Interaction method and system, and electronic device

Abstract

Disclosed in the present application are an interaction method and system, and an electronic device. The electronic device 100 can perform emotion detection on the basis of physiological data of a user to generate an emotion detection result, and generate, on the basis of the emotion detection result, and optionally in combination with personalized information of the user (e.g., environmental information of the user, exercise health data, associated event information, and personal information of the user), a personalized cartoon image capable of representing the emotion of the user, wherein the cartoon image can be displayed locally on the electronic device 100, or can be displayed on a full-scene device of the user of the electronic device 100 or other devices. In this way, social exploration requirements of users can be effectively promoted, and the users can be concerned about themselves and others. In addition, the electronic device 100 can also support the interaction between the user and the cartoon image, such that different feedbacks can be given to the user on the basis of different emotional states. In this way, the emotions of users can be effectively adjusted, thereby improving the user experience.

Description

Interaction method, system and electronic device

[0001] The present application claims priority to the Chinese patent application No. 202411307035.X, filed on September 18, 2024, and entitled "Interaction method, system and electronic device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of terminals, and in particular to an interaction method, system and electronic device. BACKGROUND

[0003] With the rapid development of terminal technology, more and more smart electronic devices (such as smart watches, smart bracelets and other wearable devices) have entered the lives of users. However, the current electronic devices lack interactivity and cannot meet the diversified needs of users (such as the need for electronic devices to help users regulate emotions). SUMMARY

[0004] The embodiments of the present application provide an interaction method, system and electronic device, which can generate and display a personalized cartoon image capable of representing the emotions of a user, and support the user to regulate emotions by interacting with the cartoon image, thereby improving interactivity.

[0005] In a first aspect, the embodiments of the present application provide an interaction method applied to a communication system including a first electronic device and a second electronic device, the method comprising: detecting, by the first electronic device, an emotion of a first user; sending, by the first electronic device, a first message to the second electronic device, the first message being used to instruct the second electronic device to display a first virtual image in a first emotion, the first emotion being the emotion of the first user detected by the first electronic device; displaying, by the second electronic device, a first interface, the first interface including the first virtual image in the first emotion; detecting, by the second electronic device, a first operation of a second user on the first virtual image; in response to the first operation, sending, by the second electronic device, a second message to the first electronic device; receiving, by the first electronic device, the second message and executing a first function based on the second message; wherein executing the first function comprises one or more of the following: outputting a second interface, outputting a first audio, running an artificial intelligence chat, and running a first game.

[0006] The second electronic device can be a remote device, the first electronic device can be a local device (i.e., an emotion detection device), the first interface can be an interface for displaying the first virtual image, which can be displayed in full screen or non-full screen (e.g., in the form of a card), the first virtual image can be a virtual image corresponding to the first user, the first user can be a user of the first electronic device, the second user can be a user of the second electronic device, the second interface can be an interface for playing a video / animation / mindfulness exercise for adjusting the user's mood, the first audio can be audio for adjusting the user's mood, and the first game can be a game (e.g., the aforementioned knocking of wooden fish or turning of a prayer bead) for adjusting the user's mood. The second interface and the first audio can be determined based on the first operation, and can be a preset soothing interface / soothing audio, or a temporarily generated soothing interface / soothing audio.

[0007] By implementing the method of the first aspect, the virtual image displayed on the local device can be displayed on the remote device, which can effectively promote the social exploration needs of the user, and the user of the remote device can give feedback to the user of the local device by operating the virtual image, adjust the mood of the user of the local device, and facilitate effective care and communication between different users.

[0008] In a possible implementation, the first operation is an operation on the first virtual image, or the first operation is an operation on a function control included in the first interface.

[0009] In a possible implementation, the first emotion is a negative emotion, the first operation is an operation for soothing the first user, and the second interface is an interface for soothing the first virtual image.

[0010] In this way, when the second user finds that the first user is in a negative emotion, the second user can give feedback to the first user by performing a soothing operation on the virtual image corresponding to the first user, so as to adjust the mood of the first user.

[0011] In a possible implementation, the second interface includes the first virtual image in a first state, and the first state corresponds to the first operation.

[0012] In a possible implementation, after the first electronic device performs the first function based on the second message, the method further includes: the first electronic device sends a third message to the second electronic device; and the second electronic device displays a third interface based on the third message, the third interface including the first virtual image in a second emotion, the second emotion being the emotion of the first user detected by the first electronic device after performing the first function.

[0013] In this way, after the emotion of the first user changes from the first emotion to the second emotion, the first electronic device can notify the second electronic device, so that the second electronic device can update the display of the first virtual image.

[0014] In a possible implementation, the first virtual image is generated based on first data, the first data being physiological data of the first user for determining the emotion; or, the first virtual image is generated based on the first data and second data, the second data being personalized information of the first user, the personalized information of the first user including one or more of the following: environmental information in which the first user is located, sports health data, event information associated with generating the first emotion, personal information of the first user.

[0015] That is, the first virtual image can be generated based on only the emotion of the user, or the first virtual image can be generated based on multiple factors (for example, the emotion of the user, environmental information in which the user is located, sports health data, event information associated with generating the current emotion, and personality type of the user).

[0016] In a possible implementation, the first interface is displayed full screen, or the first interface is displayed in the form of a card.

[0017] In this way, the first interface can be presented in multiple forms.

[0018] In a possible implementation, before the second electronic device displays the first interface, the method further includes: detecting, by the first electronic device, an operation of the first user granting the second electronic device a permission to display the first virtual image.

[0019] In this way, in the case where the first user authorizes on the first electronic device, only the electronic device (that is, the remote device) other than the first electronic device can display the first virtual image.

[0020] In a possible implementation, the first electronic device sending the first message to the second electronic device includes: in response to a first trigger event, the first electronic device sending the first message to the second electronic device; wherein the first trigger event includes one or more of the following: a change in the emotion detection result of the first user meeting a first preset condition; a change in the environmental information in which the first user is located meeting a second preset condition; a change in the sports health data of the first user meeting a third preset condition; the associated event information corresponding to the first user meeting a fourth preset condition.

[0021] In a second aspect, an embodiment of the present application provides an interaction method applied to a second electronic device, the method comprising: receiving, by the second electronic device, a first message sent by a first electronic device; in response to the first message, displaying, by the second electronic device, a first interface, the first interface including a first virtual image in a first emotion, the first emotion being an emotion of a first user detected by the first electronic device; detecting, by the second electronic device, a first operation of a second user on the first virtual image; in response to the first operation, sending, by the second electronic device, a second message to the first electronic device, the second message being used to instruct the first electronic device to perform a first function; wherein performing the first function comprises one or more of the following: outputting a second interface, outputting a first audio, running an artificial intelligence chat, and running a first game.

[0022] In a possible implementation, the first operation is an operation on the first virtual image; or, the first operation is an operation on a function control included in the first interface.

[0023] In a possible implementation, the first emotion is a negative emotion, the first operation is an operation for comforting the first user, and the second interface is an interface for comforting the first virtual image.

[0024] In a possible implementation, the second interface includes the first virtual image in a first state, the first state corresponding to the first operation.

[0025] In a possible implementation, after the second electronic device sends the second message to the first electronic device, the method further comprises: receiving, by the second electronic device, a third message sent by the first electronic device; displaying, by the second electronic device, a third interface based on the third message, the third interface including the first virtual image, the first virtual image being in a second emotion, the second emotion being an emotion of the first user detected by the first electronic device after performing the first function.

[0026] In a possible implementation, the first virtual image is generated based on first data, the first data being physiological data of the first user used to determine the emotion; or, the first virtual image is generated based on the first data and second data, the second data being personalized information of the first user, the personalized information of the first user including one or more of the following: environmental information of the first user, sports health data, event information associated with generating the first emotion, and personal information of the first user.

[0027] In a possible implementation, the first interface is displayed full screen, or the first interface is displayed in the form of a card.

[0028] In a possible implementation, the second electronic device is granted a permission to display the first virtual image.

[0029] In a possible implementation, the permission of displaying the first virtual image includes: displaying the permission of displaying the first virtual image on the fourth interface, the fourth interface including one or more of a negative one screen interface, a lock screen interface, a desktop, an off-screen display interface, and an application interface; and / or displaying the permission of displaying the first virtual image in the first time period.

[0030] In this way, the first user can autonomously set which interfaces and time periods on the remote device to display the first virtual image.

[0031] In a third aspect, an embodiment of the present application provides an interaction method applied to a first electronic device, the method comprising: detecting, by the first electronic device, an emotion of a first user; sending, by the first electronic device, a first message to a second electronic device, the first message being used to instruct the second electronic device to display a first virtual image in a first emotion, the first emotion being the emotion of the first user detected by the first electronic device; receiving, by the first electronic device, a second message sent by the second electronic device; and executing, by the first electronic device, a first function based on the second message; wherein the execution of the first function includes one or more of the following: outputting a second interface, outputting a first audio, running an artificial intelligence chat, and running a first game.

[0032] In a possible implementation, the second interface includes the first virtual image in a first state, the first state being determined based on the second message.

[0033] In a possible implementation, before the first electronic device sends the second message to the second electronic device, the method further includes: obtaining, by the first electronic device, first data, and generating the first virtual image based on the first data, the first data being physiological data of the first user used to determine the emotion; or obtaining, by the first electronic device, first data and second data, and generating the first virtual image based on the first data and the second data, the second data being personalized information of the first user, the personalized information of the first user including one or more of the following: environmental information of the first user, sports health data, event information associated with the generation of the first emotion, and personal information of the first user.

[0034] In a possible implementation, before the first electronic device sends the first message to the second electronic device, the method further includes: detecting, by the first electronic device, an operation of the first user granting the second electronic device the permission of displaying the first virtual image.

[0035] In a possible implementation, the operation of the first electronic device detecting that the first user grants the second electronic device the permission to display the first virtual image includes: the first electronic device detecting that the first user grants the second electronic device the permission to display the first virtual image on a first interface, the first interface including one or more of a negative one screen interface, a lock screen interface, a desktop, an off-screen display interface, and an application interface; and / or, the first electronic device detecting that the first user grants the second electronic device the permission to display the first virtual image in a first time period.

[0036] In a possible implementation, the method further includes: the first electronic device obtaining third data, the third data including one or more emotions of the first user in a second time period and factors associated with the one or more emotions, the factors including one or more of an activity track, schedule information, a body movement, an expression, and a speech frequency of the first user; and the first electronic device determining a personality type of the first user based on the third data.

[0037] In a possible implementation, the method further includes: the first electronic device displaying the first virtual image in a second emotion, the second emotion being an emotion of the first user detected by the first electronic device; the first electronic device detecting a first operation of the first user on the second virtual image; and in response to the first operation, the first electronic device performing a second function, the performance of the second function being based on a first emotion regulation strategy, the first emotion regulation strategy being determined from a plurality of emotion regulation strategies based on a first condition, the first condition including one or more of: the first emotion regulation strategy being effective for the first user, the personality type of the first user being a first personality type, and the first user being in a first scenario; and wherein the effectiveness includes: after the performance of the second function, the first electronic device detecting that the emotion of the first user changes to a third emotion, the third emotion being a positive emotion relative to the second emotion.

[0038] In a possible implementation, the first emotion regulation strategy includes one or more of: outputting a comfort interface, outputting a comfort audio, running an artificial intelligence chat, and running a game.

[0039] In a possible implementation, the method further includes: the first electronic device predicting that a first event will occur at a first time, the first event being a factor triggering the first user to generate a fourth emotion; and the first electronic device outputting a first prompt information, the first prompt information being used to prompt the first user to regulate the emotion.

[0040] In this way, the emotion of the user can be predicted when a certain event is about to occur, and the user can be given a relevant prompt in advance, so as to regulate the emotion in time.

[0041] In a possible implementation, the first electronic device sending the first message to the second electronic device comprises: in response to a first trigger event, the first electronic device sending the first message to the second electronic device; wherein the first trigger event comprises one or more of the following: a change in the emotion detection result of the first user meeting a first preset condition; a change in the environmental information in which the first user is located meeting a second preset condition; a change in the motion health data of the first user meeting a third preset condition; the associated event information corresponding to the first user meeting a fourth preset condition.

[0042] In a fourth aspect, the embodiments of the present application provide a mood display method, comprising: a first electronic device detecting a situation of a first user; when a first trigger condition is met, the first electronic device displaying a first interface, the first interface comprising a first virtual image in a first state; the first virtual image in the first state being used to indicate a first mood of the first user when the first trigger condition is met; when a second trigger condition is met, the first electronic device displaying a second interface, the second interface comprising a first virtual image in a second state; the first virtual image in the second state being used to indicate a second mood of the first user when the second trigger condition is met. Wherein the first trigger condition and the second trigger condition occur at different times, and the contents of the two trigger conditions can be the same or different, such as the two trigger conditions can both be wrist lifting and screen brightening, or one is wrist lifting and screen brightening and the other is crown pressing and screen brightening or monitoring the change of the mood of the user. The electronic device can dynamically display the virtual image representing the current mood of the user based on the mood of the user.

[0043] In a possible implementation, the first interface presents a first color corresponding to the first mood; and the second interface presents a second color corresponding to the second mood. When displaying the virtual image, the electronic device can display the color corresponding to the mood as the color or background color of the virtual image, so that the user can see the color and more clearly understand the mood represented by the virtual image.

[0044] In a possible implementation, when the first mood is different from the second mood, the first state is different from the second state; or when the first mood is different from the second mood, the first color is different from the second color. When the user is in different moods, the electronic device displays the virtual image in different states or displays the interface in different colors.

[0045] In a fifth aspect, the embodiments of the present application provide a chip system, comprising: a processor, the processor being coupled with a memory, the memory being used to store programs or instructions, when the programs or instructions are executed by the processor, the chip system executes the method in any possible implementation of the second aspect or the third aspect or the fourth aspect.

[0046] In a sixth aspect, an electronic device is provided, which includes one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and are configured to store computer program codes, the computer program codes including computer instructions, which, when executed by the one or more processors, cause the electronic device to perform the method in any possible implementation of the second aspect or the third aspect or the fourth aspect.

[0047] In a seventh aspect, a computer storage medium is provided, which stores a computer program, the computer program including program instructions, which, when executed on an electronic device, cause the electronic device to perform the method in any possible implementation of the second aspect or the third aspect or the fourth aspect.

[0048] In an eighth aspect, a computer program product is provided, which, when executed on a computer, cause the computer to perform the method in any possible implementation of the second aspect or the third aspect or the fourth aspect. BRIEF DESCRIPTION OF DRAWINGS

[0049] FIG. 1 is a schematic diagram of an architecture of a communication system according to an embodiment of the present application;

[0050] FIG. 2A is a flowchart of a method for generating an emotional cartoon figure according to an embodiment of the present application;

[0051] FIG. 2B is a schematic diagram of generating an emotional cartoon figure according to an embodiment of the present application;

[0052] FIG. 2C is a schematic diagram of generating an emotional cartoon figure according to another embodiment of the present application;

[0053] FIGS. 3A-3H are a set of schematic diagrams of generated emotional cartoon figures according to embodiments of the present application;

[0054] FIG. 4A is a flowchart of a method for displaying an emotional cartoon figure according to an embodiment of the present application;

[0055] FIG. 4B is a schematic diagram of displaying an emotional cartoon figure according to an embodiment of the present application;

[0056] FIGS. 5A-5K are a set of schematic diagrams of user interfaces involved in displaying settings of an emotional cartoon figure according to embodiments of the present application;

[0057] FIGS. 6A-6H are a set of schematic diagrams of user interfaces involved in displaying an emotional cartoon figure on different devices according to embodiments of the present application;

[0058] FIG. 7A is a flowchart of a personality type inference, emotion prediction and adjustment method according to an embodiment of the present application;

[0059] FIG. 7B is a schematic diagram of a personality type inference, emotion prediction and adjustment method according to an embodiment of the present application;

[0060] FIG. 8A is a flowchart of a personalized emotion adjustment strategy generation method according to an embodiment of the present application;

[0061] FIG. 8B is a schematic diagram of a personalized emotion adjustment strategy generation method according to an embodiment of the present application;

[0062] FIG. 8C is a schematic diagram of a personalized emotion adjustment strategy recommendation method according to an embodiment of the present application;

[0063] FIGS. 9A-9C are a set of user interface diagrams for emotion adjustment through video / animation playing according to an embodiment of the present application;

[0064] FIGS. 10A-10B are a set of user interface diagrams for emotion adjustment through audio playing according to an embodiment of the present application;

[0065] FIGS. 11A-11B are a set of user interface diagrams for emotion adjustment through talking according to an embodiment of the present application;

[0066] FIGS. 12A-12D are a set of user interface diagrams for emotion adjustment through mindfulness practice according to an embodiment of the present application;

[0067] FIGS. 13A-13D are a set of user interface diagrams for emotion adjustment through small games according to an embodiment of the present application;

[0068] FIG. 14 is a user interface diagram for providing emotion adjustment service according to an embodiment of the present application;

[0069] FIGS. 15A-15G are a set of user interface diagrams for emotion adjustment of a user of a remote device according to an embodiment of the present application;

[0070] FIG. 16 is a schematic diagram of an interaction method according to an embodiment of the present application;

[0071] FIGS. 17A-17H are schematic diagrams of another emotion display and interaction method according to an embodiment of the present application;

[0072] FIG. 18 is a structural diagram of an electronic device 100 according to an embodiment of the present application;

[0073] FIG. 19 is a software structure diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0074] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, “ / ” represents or, for example, A / B can represent A or B; “and / or” in the text only represents a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone. In addition, in the description of the embodiments of the present application, “multiple” means two or more than two.

[0075] It should be understood that the terms “first”, “second” and the like in the description and claims of the present application and the drawings are used to distinguish different objects, and are not used to describe a specific order. In addition, the terms “include” and “have” and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0076] In the present application, “embodiment” means that the specific features, structures or characteristics described in combination with the embodiment can be included in at least one embodiment of the present application. The phrase appears at various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment to other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in the present application can be combined with other embodiments.

[0077] In the following embodiments of the present application, the term “user interface (UI)” is a medium interface for interaction and information exchange between an application or an operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is source code written in a specific computer language such as Java, extensible markup language (XML), etc. The interface source code is parsed, rendered and finally presented as content recognizable by the user on the electronic device. The commonly used form of user interface is graphic user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be a visual interface element such as text, icon, button, menu, tab, text box, dialog box, status bar, navigation bar, Widget, etc. displayed in the display screen of the electronic device.

[0078] Currently, the display content of many intelligent electronic devices (for example, wearable devices such as smart watches, smart bands, and the like) on the watch face, theme, service card, and the like interface is unchangeable or cycled, and the interface design lacks interactivity, which cannot meet the diversified interactive needs of users. For example, in the case that a user unknowingly immerses in a negative emotion, the electronic device cannot effectively remind and intervene to help the user to adjust the emotion; for another example, in a deep social scene such as parent-child, lovers, and friends, the user cannot know the emotion of the other party through the electronic device, and thus cannot effectively communicate based on the emotion of the other party.

[0079] Based on the above problems, an embodiment of the present application provides an interaction method. The electronic device 100 can perform emotion detection based on physiological data of a user, generate an emotion detection result, and based on the emotion detection result, generate a personalized virtual image capable of representing the emotion of the user, optionally in combination with personalized information (for example, environmental information, motion health data, associated event information, personal information of the user, and the like) of the user. The above virtual image can be displayed on the electronic device 100, or on a full-scene device (that is, other devices logged in to the same account as the local device) of the user of the electronic device 100 or a device of another person, which can effectively promote the social exploration needs of the user and facilitate the user to effectively care for themselves and others. In addition, the electronic device 100 can also support the user to interact with the above virtual image, so that different feedback results can be given to the user based on different emotional states, which can effectively regulate the emotion of the user and improve the user experience.

[0080] The above method can be applied to various scenarios. For example, in a vehicle-mounted scenario, the emotion of a driver can be detected during driving, and the emotion of the driver can be relieved in time when the driver has a negative emotion or a high stress emotion, thereby improving the safety during driving; for another example, in a smart home scenario, the user can be guided to adjust the emotion by using a massage chair, a treadmill, and the like home device based on the emotion of the user, and further, the emotional changes of the user before and after use can be compared to evaluate the emotional improvement effect; for another example, in a game scenario, the emotion of a player can be detected in real time, and the difficulty of the game can be automatically adjusted based on the emotion of the player; for another example, in a child education and care scenario, the emotional state of a child or an old person can be detected, and psychological counseling can be given to the child or the old person when necessary; for another example, in a digital human emotional accompanying scenario, the digital human can be online for 24 hours, and the digital human can chat in a personified manner, and the digital human can provide emotional healing services to the user when detecting that the user has a poor emotion; for another example, in a high-pressure industry population care scenario, the emotional change trend of a high-pressure industry population can be monitored in real time, and psychological counseling can be given to the high-pressure industry population when necessary; and the like.

[0081] It should be noted that the virtual image (which can also be referred to as a role image) mentioned in the embodiments of the present application can be various types of images, for example, a cartoon image, which can be, for example, a pet image, a human image, a plant image, and the like. The specific type of virtual image is not limited in the embodiments of the present application. The virtual image can be different. For example, the virtual image can be a panda or a puppy. It can be understood that the panda and the puppy belong to different virtual images.

[0082] For ease of understanding, some related concepts involved in the embodiments of the present application are described below.

[0083] 1, emotion (or emotion state)

[0084] Emotion is a kind of psychological activity mediated by the subject's desire and need. Emotion is usually composed of three components: unique subjective experience, external manifestation and physiological arousal.

[0085] 2, emotion model classification

[0086] Commonly used emotion models are mainly divided into two categories: continuous emotion model and discrete emotion model.

[0087] The continuous emotion model can be understood from the perspective of dimension. The continuous emotion model is based on multi-dimensional theory and classifies emotions in multiple dimensions or scales.

[0088] The continuous emotion model can describe emotions as points in a multi-dimensional emotion space. Each dimension in the space corresponds to a psychological attribute of emotion. Almost all emotions can find corresponding mapping points in the emotion space.

[0089] The dimensional space model of emotion can be divided into different types such as two dimensions and three dimensions.

[0090] The commonly used continuous emotion model is the two-dimensional representation model of emotion. The horizontal axis and the vertical axis of the model represent valence and arousal respectively.

[0091] Among them, the valence (or valence) can represent the degree of pleasure of emotion (or the degree of positive / negative of emotion). It is easy to understand that when the needs of a person are met, positive emotions (or positive emotions) are usually generated, and the activity ability (or dynamic nature) of the person is improved; when the needs of a person are not met, negative emotions (or negative emotions) are usually generated, and the activity ability of the person is reduced.

[0092] Among them, the arousal can represent the intensity of emotion. The intensity of emotion can have two ends of strong and weak, for example, from happy to ecstatic, from slightly angry to furious.

[0093] Exemplarily, in the above-mentioned two-dimensional emotion representation model, happy can be represented by high valence and high arousal, and sad can be represented by low valence and low arousal.

[0094] The discrete emotion model can be understood from the perspective of category. The discrete emotion model is based on the basic emotion theory, and marks emotions as discrete categories.

[0095] The discrete emotion model can describe emotions as discrete (basic) and adjective-labeled forms, such as joy, anger, pity, happiness, sorrow, fear, surprise, and the like.

[0096] 3. Emotion detection

[0097] Emotion detection refers to assessing and measuring the emotional state of an individual through one or more ways.

[0098] There can be various ways of emotion detection, for example, emotion detection can be performed by acquiring features of physiological signals (or physiological data); for another example, emotion detection can be performed by acquiring features of facial expressions or speech signals; and the like.

[0099] Since the change of human emotion is usually accompanied by the change of physiological signals, the physiological signals have the advantage of reflecting the true emotion of a person compared to facial expressions or speech signals, while facial expressions and speech signals are not fine enough to represent emotions and are easy to fake and involve a higher level of user privacy.

[0100] In the embodiments of the present application, emotion detection can be performed based on the physiological signals of the user collected.

[0101] Generally, the physiological signals used for emotion detection include electrocardiogram signals, electroencephalogram signals, electromyogram signals, eye movement signals, respiration signals, electrodermal signals, blood pressure signals, blood oxygen saturation, pulse signals, and the like.

[0102] Exemplarily, in emotion detection, a PPG sensor, an electrocardiogram sensor, or the like can be used to collect the original electrocardiogram signals of the user, and based on the electrocardiogram signals, the R-R interval (RRI) data of the user in a period of time can be obtained, further, the RRI data can be processed to obtain features such as heart rate variability (HRV), sympathetic activity index (SAI), and parasympathetic activity index (PAI), and based on these features, a valence model can be established to obtain an emotion classification result, so as to determine the current emotion of the user.

[0103] In the embodiments of the present application, the physiological signal collected is taken as an example of an electrocardiogram for emotion detection, which should not be construed as a limitation of the present application. In some other embodiments of the present application, other ways can also be used for emotion detection.

[0104] 4. Emotion regulation

[0105] Emotion regulation refers to the process of managing and changing the emotional state of an individual through certain strategies and mechanisms, so that the individual's emotions change in physiological activity, subjective experience, expression behavior, etc.

[0106] There are many ways to regulate emotions. For example, at the level of relaxation exercises, emotion regulation can be achieved through breathing exercises, mindfulness meditation, etc. For example, at the lifestyle level, emotion regulation can be achieved through exercise, sleep, balanced diet, reasonable sunlight, etc. For example, at the physical stimulation level, emotion regulation can be achieved through aromatherapy, massage, eating delicious food, listening to music, art appreciation, watching videos, playing games, traveling, puzzles, coloring, fish keeping, ceramics making, etc. For example, at the cognitive level, emotion regulation can be achieved through chatting, throwing a message in a bottle, learning psychological knowledge, instilling spiritual chicken soup, writing a diary, etc.

[0107] Next, a communication system provided by an embodiment of the present application is introduced.

[0108] FIG. 1 exemplarily shows a communication system provided by an embodiment of the present application.

[0109] As shown in FIG. 1, the communication system can include an electronic device 100 (a local device), an electronic device 200 (a remote device).

[0110] Among them, the electronic device 100 and the electronic device 200 are both intelligent terminal devices, which can be of various types, and the embodiments of the present application do not limit them.

[0111] For example, the electronic device 100 can be a watch, a bracelet, etc.

[0112] In the embodiments of the present application, the electronic device 100 can be used to generate and display an emotional cartoon image, which can represent the user's emotions; can also be used to share the above emotional cartoon image to the user's full-scene device or other people's devices of the electronic device 100 based on short-distance communication or cloud communication, etc., so that the above emotional cartoon image can also be displayed on the user's full-scene device or other people's devices of the electronic device 100; can also be used to give different feedback results to the user through the above emotional cartoon image based on different emotional states of the user; etc.

[0113] For example, the electronic device 200 can be a mobile phone, a large screen, a tablet, a personal computer, a car machine, a watch, a bracelet, etc.

[0114] The electronic device 200 can be a full-scene device of a user of the electronic device 100, or can be a device of another person, for example, a device of a contact (for example, a family member, a friend, or the like) of the user of the electronic device 100.

[0115] In the embodiments of the present application, the electronic device 200 can be used to receive and display the emotional cartoon image shared by the electronic device 100.

[0116] The electronic device 100 and the electronic device 200 in the communication system can be connected by logging in the same account. For example, the same Huawei account can be logged in, and the server can be used to remotely connect and communicate. The account can be a mobile phone number, an email number, a custom username, an APP server assigned username, a username used in a certain APP in the electronic device, or the like.

[0117] The electronic device 100 and the electronic device 200 in the communication system can also log in different accounts. In some examples, the electronic devices can be connected by binding. After one electronic device logs in an account, the other electronic devices that log in different accounts or do not log in can be bound in a device management application, and then the electronic devices can communicate through the device management application. For example, the electronic devices can be multiple devices of family members, or multiple devices of a friend / family group. In other examples, the electronic device 200 can also be a device of another contact of the user of the electronic device 100. In the embodiments of the present application, the account in each embodiment is not limited, for example, the account can be independent of a mobile phone number / device identifier, or can be bound with the mobile phone number / device identifier, or the mobile phone number / device identifier can be directly used as the account.

[0118] The multiple electronic devices in the communication system can also be connected by scanning a two-dimensional code, near field communication (NFC) touch, searching for a Bluetooth device, or the like, which is not limited herein.

[0119] In general, the communication connection established between the electronic device 100 and the electronic device 200 in the communication system can include, but is not limited to, a mobile network connection (such as 4G, 5G, etc.), a short-distance connection such as a Bluetooth (BT) connection, a wireless local area network (WLAN) such as a wireless fidelity point to point (Wi-Fi P2P) connection, a near field communication (NFC) connection, an infrared (IR) connection, or a wired connection, and the like.

[0120] In addition, the electronic device 100 and the electronic device 200 in the communication system can also be connected and communicate in combination with any of the above-mentioned manners, and the embodiments of the present application do not limit this.

[0121] It should be understood that FIG. 1 is only a schematic diagram of the architecture of the communication system and should not be construed as a limitation of the present application. The number of electronic devices 100 and electronic devices 200 included in the communication system is not limited in the embodiments of the present application.

[0122] Embodiment one, emotion cartoon image generation

[0123] FIGS. 2A-2C exemplarily show a specific flow of an emotion cartoon image generation method provided in the embodiments of the present application.

[0124] Referring to FIGS. 2A-2C, the method can be applied to the electronic device 100, and the specific steps of the method are described in detail as follows:

[0125] S201, the electronic device 100 collects physiological data (such as heart rate interval RRI data) of the user, and performs emotion detection based on the collected physiological data to generate an emotion detection result.

[0126] In the embodiments of the present application, emotion detection can be performed based on the physiological data of the user, which should not be construed as a limitation of the present application. In some embodiments of the present application, emotion detection can also be performed based on facial expressions and the like of the user. The physiological data can include, but is not limited to, electrocardiogram data, electroencephalogram data, electromyogram data, eye movement data, respiration data, electrodermal data, blood pressure data, blood oxygen saturation, and pulse data.

[0127] Exemplarily, the electronic device 100 can include a photoplethysmography (PPG) sensor (or PPG module), which can collect physiological data of a user, for example, heart rate interval (RRI) data. After the data collection is completed, the data can be input into an emotion detection algorithm model for processing. After the processing is completed, the emotion detection algorithm model can output an emotion detection result, which can be used to indicate the emotional state of the user. Taking electrocardiogram data as an example, after the user 1 wears the electronic device 100, the electronic device 100 can periodically collect electrocardiogram data of the user 1. For example, the electronic device 100 can collect 90 seconds of heart rate interval (R-R interval, RRI) data every 10 minutes. Based on the RRI data, multiple features required for emotion detection can be obtained (for example, through an autonomic nervous sequence decomposition algorithm), such as heart rate variability (HRV) frequency domain features, HRV time domain features, sympathetic sequences, parasympathetic sequences, and the like. Based on the multiple features, a valence model or an arousal model can be established, and then an emotion detection result (for example, three-class positive emotions (such as joy), negative emotions (such as unhappiness), and neutral emotions (such as calmness)) can be obtained.

[0128] In some examples, before performing the above step S201, the electronic device 100 can further receive a user operation for granting the electronic device 100 the permission for emotion detection.

[0129] It should be noted that the above emotion detection process can be performed on the electronic device 100 or on a server, and the present application does not limit the same.

[0130] It should be noted that in the present application, the emotion detection result can include the belonging emotion category of the emotional state of the user. The emotion category can be obtained by classifying the emotion in N dimensions using the aforementioned continuous emotion model (where N can be a positive integer greater than 1, such as N = 3 or 5 or 7, i.e., three-class, five-class or seven-class classification of emotions), or can be obtained by marking the emotion as discrete categories such as joy, anger, sorrow, pleasure, sadness, fear, and surprise using the aforementioned discrete emotion model, and the present application does not limit the same.

[0131] S202, the electronic device 100 generates an emotional cartoon image 1 based on the emotion detection result.

[0132] The above emotional cartoon image 1 can be a cartoon image that can represent the emotional state of the user.

[0133] In the present application, a default emotional cartoon image can be generated based on the emotion detection result.

[0134] In a possible implementation, a mapping table can be stored, in which each emotion can correspond to one or more default emotion cartoon images. First, the emotion of the user can be determined based on the emotion detection result, and then the emotion cartoon image corresponding to the emotion can be found from the mapping table. Taking the smart watch as an example, it is assumed that the emotion detection result shows that the emotion of the user is calm, and referring to FIG. 3A, FIG. 17D or FIG. 17E, the emotion cartoon image can be the emotion cartoon image 1, that is, a default emotion cartoon image in a calm emotion generated based on the emotion detection result. For another example, when the emotion of the user is happy, the electronic device 100 can display the interface shown in FIG. 17B or FIG. 17C. When the emotion of the user is unhappy, the electronic device 100 can display the interface shown in FIG. 17F or (1) of FIG. 17H.

[0135] In another possible implementation, the emotion can also be represented by a virtual image and a color. For example, yellow or orange represents happiness, green represents calm, and blue or purple represents unhappiness. When the electronic device 100 generates the emotion cartoon image 1 based on the emotion detection result in S202, the electronic device 100 can generate a cartoon image with a color corresponding to the emotion detection result, or set the background color of the cartoon image to correspond to the emotion detection result, or display the cartoon image with a background color corresponding to the emotion detection result. For example, the interface shown in FIG. 17B or FIG. 17C can be yellow or orange; the interface shown in FIG. 3A, FIG. 17D or FIG. 17E can be green; the interface shown in FIG. 17F or (1) of FIG. 17H can be blue or purple; one or more of (2)-(8) of FIG. 17G and (2)-(6) of FIG. 17H can be yellow or orange, or green.

[0136] It should be noted that the process of generating the emotion cartoon image 1 can be performed on the electronic device 100 or on a server, and the embodiments of the present application do not limit this.

[0137] In the embodiments of the present application, the emotion cartoon image is generated based on the result of real-time emotion detection. In some examples, the electronic device 100 can also support the user to manually input his / her emotion, and generate an emotion cartoon image based on the emotion, or support the user to manually change the result of real-time emotion detection, and generate an emotion cartoon image based on the changed result.

[0138] In S203, the electronic device 100 acquires personalized information of the user (for example, environment information of the user, motion health data, associated event information, personal information of the user, etc.).

[0139] The personalized information of the user can include one or more of the following information: environmental information of the user, motion health data, associated event information, and personal information of the user.

[0140] The environmental information can include one or more of the following information: weather, time, festival, location, and the like.

[0141] The motion health data can include one or more of the following information: sleep data (e.g., sleep-in and sleep-out data), motion data (e.g., a hundred kinds of motion data), breathing training data (e.g., breathing guidance data), health data (e.g., heart rate, blood oxygen, body temperature, blood pressure, and the like), and the like.

[0142] The associated event information can include one or more of the following information: social activity data (e.g., work, study, housework, dating, and the like), personality and hobby data (e.g., fitness, travel, solitude, and the like), interpersonal relationship data (e.g., partner, neighborhood, friend, family, and the like), and life routine data (e.g., listening to music, eating, drinking milk tea, playing games, driving, taking a car, exercising, and the like), and the like. In the embodiments of the present application, the associated event information can be obtained in various ways, for example, can be manually input by the user, or can be obtained through device data (e.g., schedule data or application data), and the embodiments of the present application are not limited thereto.

[0143] The personal information of the user can include one or more of the following user information: age, gender, interest, and user portrait of the user. In a possible implementation manner, the personal information of the user can be obtained according to the account information of the user.

[0144] In some examples, the personal information of the user can further include appearance feature information (e.g., one or more of the following feature information: clothing, accessories, body shape, and facial features) of the user. In a possible implementation manner, the appearance feature information of the user can be extracted or generated from a photo taken by the user or a picture in a gallery uploaded by the user.

[0145] In other examples, the personal information of the user can further include personality type information of the user. In a possible implementation manner, the personality type of the user can be animalized, for example, the personality type can include panda type, tiger type, peacock type, and the like.

[0146] In some examples, before performing the step S203, the electronic device 100 can optionally further include: receiving a user operation for granting the electronic device 100 the permission to obtain the personalized information of the user.

[0147] It should be noted that, in the embodiments of the present application, if the electronic device 100 is not granted the permission to obtain the personalized information of the user, the electronic device 100 can only generate the above-mentioned emotional cartoon image 1 (i.e. the default emotional cartoon image), without the need to perform steps S203 and S204. Alternatively, if the electronic device 100 is only granted to obtain part of the personalized information of the user, the emotional cartoon image 1 can be generated using the granted part of the personalized information and the emotional information, for example, as shown in FIG. 2B, the part of the personalized information can refer to the user information and the generated picture / photo, and the emotional cartoon image 1 can be generated based on the user information and the generated picture / photo and the emotional detection result.

[0148] It should be noted that the embodiments of the present application do not limit the execution time sequence of the above-mentioned steps S201-S202 and the above-mentioned step S203, for example, the above-mentioned steps S201-S202 can be executed before or after the above-mentioned step S203; for another example, the above-mentioned steps S201-S202 and the above-mentioned step S203 can also be executed simultaneously.

[0149] S204, the electronic device 100 generates an emotional cartoon image 2 based on the emotional cartoon image 1 and the personalized information of the user, or based on the emotional detection result and the personalized information of the user.

[0150] Among them, the above-mentioned emotional cartoon image 2 (which can also be called an emotional cartoon image of superimposed scene linkage) can be a cartoon image that can represent the emotional state of the user and can represent the personalized characteristics of the user.

[0151] In one possible implementation, for example, as shown in FIG. 2C, the above-mentioned emotional cartoon image 2 can be generated based on the generated emotional cartoon image 1, in combination with one or more of the environmental information, the sports health data, the associated event information, and the personal information of the user included in the above-mentioned personalized information of the user.

[0152] In another possible implementation, the above-mentioned step S202 can not be executed, but the emotional cartoon image 2 can be directly generated based on the emotional detection result generated in the above-mentioned step S201 and the personalized information of the user obtained in the above-mentioned step S203.

[0153] The following exemplary describes the generation of the emotional cartoon image 2 in combination with the environmental information of the user, or the sports health data, or the associated event information.

[0154] 1. In combination with the environmental information of the user

[0155] Exemplarily, referring to FIG. 3B, taking the environmental information as weather information as an example, in some examples, the background of the emotional cartoon image 2 can reflect the weather of the current location of the user, for example, if the weather of the current location of the user is raining, then the background of the emotional cartoon image 2 can be the background of a rainy day as shown in FIG. 3B. In other examples, the subject of the emotional cartoon image 2 can also reflect the weather of the current location of the user, for example, if the weather of the current location of the user is raining, then the emotional cartoon image 2 can hold an umbrella as shown in FIG. 3B, or wear a raincoat, and the like.

[0156] Exemplarily, referring to FIG. 3C, taking the environmental information as location information as an example, in some examples, the background of the emotional cartoon image 2 can reflect the location of the current location of the user, for example, if the location of the current location of the user is the home of the user, then the background of the emotional cartoon image 2 can be the background of a living room or a bedroom as shown in FIG. 3C, or the emotional cartoon image 2 can wear a home clothes, and the like.

[0157] 2, in combination with sports health data

[0158] Exemplarily, referring to FIG. 3D, taking the sports health data as sleep data as an example, assuming that the sleep data shows that the user usually falls asleep at a certain time point, but the user does not enter a sleep state when reaching the time point, then an emotional cartoon image 2 as shown in FIG. 3D that is sleeping can be generated to prompt the user to fall asleep.

[0159] Exemplarily, referring to FIG. 3E, taking the sports health data as exercise data as an example, assuming that the exercise data shows that the user is in an exercise state, then an emotional cartoon image 2 as shown in FIG. 3E that is exercising can be generated. In some examples, the location information of the user when exercising can also be obtained, and based on the location information, it can be determined which exercise the user is doing, so that an emotional cartoon image 2 that is more in line with the real exercise situation of the user can be generated, for example, in the case that the obtained location is a basketball court, an emotional cartoon image 2 that is playing basketball can be generated; for example, in the case that the obtained location is a swimming pool, an emotional cartoon image 2 that is swimming can be generated.

[0160] Exemplarily, referring to FIG. 3F, taking the sports health data as body temperature data as an example, assuming that the body temperature data shows that the body temperature of the user exceeds the normal body temperature value, then an emotional cartoon image 2 as shown in FIG. 3F that is in a fever state can be generated.

[0161] 3, in combination with associated event information

[0162] For example, referring to FIG. 3G, taking the life routine event of eating as an example, based on the associated event information, it can be determined that the user is eating, and then an emotion cartoon image 2 as shown in FIG. 3G, for example, of the user eating with chopsticks can be generated.

[0163] For example, referring to FIG. 3H, taking the life routine event of working as an example, based on the associated event information, it can be determined that the user is working, and then an emotion cartoon image 2 as shown in FIG. 3H, for example, of the user working with a computer can be generated.

[0164] In some examples, the above-mentioned emotion cartoon image 2 can also be generated in combination with at least two of the environment information of the user, the motion health data, and the associated event information. For example, the environment information of the user shows that the user is at the seaside, and the motion health data shows that the user is running, and then the above-mentioned emotion cartoon image 2 can be an image of the user running at the seaside; for another example, the environment information of the user shows that the weather is sunny and the user is in a park, and the associated event information shows that the user is listening to music, and then the above-mentioned emotion cartoon image 2 can be an image of the user listening to music in the park on a sunny day; for another example, the motion health data shows that the user is running, and the associated event information shows that the user is listening to music, and then the above-mentioned emotion cartoon image 2 can be an image of the user running while listening to music; for another example, the environment information of the user shows that the user is at home, the motion health data shows that the user is doing yoga, and the associated event information shows that the user is listening to music, and then the above-mentioned emotion cartoon image 2 can be an image of the user doing yoga at home while listening to music.

[0165] It should be noted that the process of generating the above-mentioned emotion cartoon image 2 can be performed on the electronic device 100 or on a server, and the embodiments of the present application do not limit this.

[0166] It should be noted that in the embodiments of the present application, the generated emotion cartoon image (for example, the above-mentioned emotion cartoon image 1 and the above-mentioned emotion cartoon image 2) can be displayed in various forms, such as animation form, picture form, etc., and the embodiments of the present application do not limit this.

[0167] By implementing the method provided in the embodiment shown in FIG. 2A, a personalized cartoon image capable of representing the user's emotion can be generated based on the emotion detection result and the user's personalized information (which can also be referred to as a scene factor), compared with a boring curve or a fixed schematic diagram, the flexible and variable emotion cartoon image not only can more vividly reflect the user's emotion, but also can significantly improve the interest of device use, and provide a better device use experience for the user.

[0168] Embodiment two, emotion cartoon image display

[0169] After the emotional cartoon image is generated, the emotional cartoon image can be displayed. The emotional cartoon image can be displayed on the local device, or can be displayed on a remote device (e.g., a full-scene device of the user of the local device or a device of another person).

[0170] In a possible implementation, after the emotional cartoon image is generated, the emotional cartoon image can be saved on the local device and can be directly displayed on the local device.

[0171] In another possible implementation, after the emotional cartoon image is generated, the emotional cartoon image can be uploaded to a server (or a data platform), and the emotional cartoon image can be displayed on the local device and the remote device. In some examples, the remote device can display the emotional cartoon image only when the user of the local device grants the remote device the permission to display the emotional cartoon image.

[0172] In another possible implementation, the local device can also send the emotional cartoon image to the remote device, so that the emotional cartoon image can be displayed on the remote device.

[0173] The specific process in which the emotional cartoon image is displayed on the remote device will be described below with reference to FIG. 4A.

[0174] FIG. 4A exemplarily shows a specific process of a method for displaying an emotional cartoon image according to an embodiment of the present application.

[0175] Referring to FIG. 4A, the method can be applied to a communication system including an electronic device 100 (a local device) and an electronic device 200 (a remote device). The specific steps of the method will be described in detail as follows.

[0176] S401. The electronic device 100 detects a setting operation of a user, and the setting operation is used to set how the emotional cartoon image is displayed on the electronic device 200.

[0177] S402. The electronic device 100 sends a message 1 to the electronic device 200 based on the setting operation, and the message 1 includes information of the emotional cartoon image.

[0178] S403. The electronic device 200 displays the emotional cartoon image based on the message 1.

[0179] Optionally, before S401 or S402, the method can further include: the electronic device 100 and the electronic device 200 establish a communication connection. In a possible implementation, the communication connection can be a short-distance communication connection established based on Bluetooth, WiFi, etc., and the electronic device 100 can perform near-field data transmission with the electronic device 200 through the communication connection. In a near-field communication scenario, the electronic device 100 and the electronic device 200 can also establish the communication connection by discovering the devices first and then performing S401.

[0180] In a possible implementation, the communication connection can be a remote communication connection established based on logging in the same account (for example, a Huawei account), and the electronic device 100 can perform cloud data transmission with the electronic device 200 through the communication connection.

[0181] In another possible implementation, the electronic device 100 can also send the message 1 to the electronic device 200 through a mobile network.

[0182] In another possible implementation, the electronic device 100 can send the message 1 to the electronic device 200 through cloud communication, for example, the user 1 of the electronic device 100 and the user 2 of the electronic device 200 are in the same group, for example, a family space, in which case the electronic device 100 can send the message 1 to the group, so that the user of the electronic device 200 in the group can receive the message 1.

[0183] In some examples, before performing S402, the method further includes: the electronic device 100 detects that a trigger condition is met; and in response to the trigger condition, the electronic device 100 performs S402. The description of the trigger condition can refer to the description of S1602 in the embodiment shown in FIG. 16.

[0184] By implementing the method provided in the embodiment shown in FIG. 4A, the emotion cartoon image displayed on the local device can be displayed on the remote device, so that in the case where the remote device is a full-scene device of the user of the local device, the user can have a full-scene experience, and the user can know his / her emotion in time in different scenes, for example, in a vehicle scene, the emotion cartoon image can be displayed on a car machine; for example, in a sports scene, the emotion cartoon image can be displayed on a wearable device such as a watch; for example, in a home scene, the emotion cartoon image can be displayed on a device such as a mobile phone or a tablet; and so on; in the case where the remote device is a device of another person, the user's social exploration demand can be effectively promoted, and effective care and communication between the user and the other person can be facilitated.

[0185] In a possible implementation, referring to FIG. 4B, after the emotional cartoon image is generated, the emotional cartoon image can be presented locally or uploaded to a data platform, and the user's own full-scene device or other device can obtain the emotional cartoon image from the data platform, and after downloading, the emotional cartoon image can be presented on one or more applications or interfaces (for example, a negative screen interface, a lock screen interface, a desktop, an off-screen display interface, an application interface, and the like). The user's own full-scene device can include but is not limited to a watch, a mobile phone, a tablet, a computer, and a car machine, and the other device can include but is not limited to a watch, a mobile phone, a tablet, and a computer.

[0186] In the embodiment of the present application, the electronic device 100 can support the user to set a display setting of the emotional cartoon image, and the display setting can refer to setting how the emotional cartoon image is displayed on the local device or the remote device. The following will be exemplarily introduced in combination with a series of user interfaces.

[0187] 1, local device display setting

[0188] Exemplarily, referring to FIG. 5A, the electronic device 100 can detect an operation (for example, a click operation) of the user on the “local display” option, and in response to the operation, the electronic device 100 can display the user interface exemplarily shown in FIG. 5B.

[0189] In the embodiment of the present application, the emotional cartoon image can be displayed in full screen on the electronic device 100, or can be displayed in non-full screen on the electronic device 100, wherein in the case of being displayed in non-full screen on the electronic device 100, the emotional cartoon image can be but is not limited to displayed in the form of a service card on the electronic device 100.

[0190] Exemplarily, referring to FIG. 5B, the user interface shown in FIG. 5B can include a “full screen” option and a “card” option, wherein the “full screen” option can be used to display the emotional cartoon image in full screen on the electronic device 100, and the “card” option can be used to display the emotional cartoon image in the form of a service card on the electronic device 100. The electronic device 100 can detect an operation (for example, a click operation) of the user on the “full screen” option, and in response to the operation, the electronic device 100 can display the emotional cartoon image in full screen; or the electronic device 100 can detect an operation (for example, a click operation) of the user on the “card” option, and in response to the operation, the electronic device 100 can display the emotional cartoon image in the form of a service card.

[0191] It should be noted that in the case of displaying the emotional cartoon image in full screen on the electronic device 100, the emotional cartoon image will not affect the display of other information (for example, time information, and the like), and in a possible implementation, the emotional cartoon image can be displayed in full screen on the electronic device 100 as the background of the other information.

[0192] 2. Remote device display setting

[0193] It is easy to understand that the remote device display setting process can refer to a process in which the user grants the remote device (e.g., the all-scene device of the user of the local device or a device of another person) the permission to display the emotion cartoon image.

[0194] By way of example, referring back to FIG. 5A, the electronic device 100 can detect an operation (e.g., a tapping operation) of the user on the “other device display” option, and in response to the operation, the electronic device 100 can display the user interface shown in FIG. 5C.

[0195] Referring to FIG. 5C, the user interface shown in FIG. 5C can include a “same account device” option and a “different account device” option, where the “same account device” option can be used to trigger setting of how the emotion cartoon image is displayed on the same account device (i.e., the all-scene device of the user of the electronic device 100), and the “different account device” option can be used to trigger setting of how the emotion cartoon image is displayed on the different account device (i.e., the device of another user, or the device of another person).

[0196] First, taking the “same account device” as an example, referring back to FIG. 5C, the electronic device 100 can detect an operation (e.g., a tapping operation) of the user on the “same account device” option, and in response to the operation, the electronic device 100 can display the user interface shown in FIG. 5D.

[0197] Referring to FIG. 5D, the user interface shown in FIG. 5D can include one or more options (e.g., a “my phone” option, a “my tablet” option, a “my car machine” option, etc.) corresponding to the devices logged into the same account as the electronic device 100, and the one or more options can be used to trigger setting of how the emotion cartoon image is displayed on the corresponding devices. Taking the “my phone” as an example, the electronic device 100 can detect an operation (e.g., a tapping operation) of the user on the “my phone” option, and in response to the operation, the electronic device 100 can display the user interface shown in FIG. 5E.

[0198] Referring to FIG. 5E, the user interface shown in FIG. 5E can include one or more setting options (e.g., a “display interface setting” option, a “display time setting” option, etc.), where the “display interface setting” option can be used to trigger setting of which interfaces the emotion cartoon image is displayed on, and the “display time setting” option can be used to trigger setting of which time period the emotion cartoon image is displayed in.

[0199] With "display interface settings" as an example, and with reference to FIG. 5E, the electronic device 100 can detect an operation (e.g., a tapping operation) of the user on the "display interface settings" option, and in response to the operation, the electronic device 100 can display the user interface shown in the example of FIG. 5F.

[0200] With reference to FIG. 5F, the user interface shown in FIG. 5F can include one or more options (e.g., a "one-screen-before interface" option, a "lock screen interface" option, a "desktop" option, an "off-screen display interface" option, and an "application interface" option), which can be used to determine which interface(s) to display the emotional cartoon image on. For example, the electronic device 100 can detect an operation (e.g., a tapping operation) of the user on the "one-screen-before interface" option, and in response to the operation, the electronic device 100 can grant the corresponding device the permission to display the emotional cartoon image on the one-screen-before interface, so that the corresponding device can display the emotional cartoon image on the one-screen-before interface.

[0201] With reference to FIG. 5F, the electronic device 100 can detect an operation (e.g., a tapping operation) of the user on the "application interface" option, and in response to the operation, the electronic device 100 can display the user interface shown in the example of FIG. 5G.

[0202] With reference to FIG. 5G, the user interface shown in FIG. 5G can include one or more options (e.g., a "sports health" option, a "smooth connection" option, and the like), which can be used to determine which interface(s) of which application to display the emotional cartoon image on. For example, the electronic device 100 can detect an operation (e.g., a tapping operation) of the user on the "sports health" option, and in response to the operation, the electronic device 100 can grant the corresponding device the permission to display the emotional cartoon image on the interface of the sports health application, so that the corresponding device can display the emotional cartoon image on the interface of the sports health application.

[0203] With "display time settings" as an example, and with reference to FIG. 5E, the electronic device 100 can detect an operation (e.g., a tapping operation) of the user on the "display time settings" option, and in response to the operation, the electronic device 100 can display the user interface shown in the example of FIG. 5H.

[0204] Referring to FIG. 5H, the user interface shown in FIG. 5H can be used to set the time period during which the emotional cartoon image is displayed. The electronic device 100 can support the user to input the corresponding time in the time period input box (including the start time input box and the end time input box) shown in FIG. 5H, for example, the start time shown in FIG. 5H is 8 o'clock and the end time is 20 o'clock, that is, the time period during which the emotional cartoon image is displayed is from 8 o'clock in the morning to 8 o'clock in the evening. In some examples, the electronic device 100 can also support the user to set the date type (for example, working day, statutory holiday, all date types, custom date type, etc.) to which the above-mentioned time period applies.

[0205] After completing the relevant settings (for example, display interface settings, display time settings) of the corresponding device (for example, "my mobile phone"), the electronic device 100 can display the user interface shown in FIG. 5I, referring to FIG. 5I, the user interface shown in FIG. 5I can include a "setting is complete and apply" option, the electronic device 100 can detect the operation (for example, click operation) of the user on the "setting is complete and apply" option, in response to the operation, the electronic device 100 can save the above-mentioned setting data, and notify the corresponding device that it has been granted the permission to display the emotional cartoon image based on the above-mentioned setting data. Further, the device can obtain and display the emotional cartoon image.

[0206] It should be noted that the above is only an example of the display setting process of the emotional cartoon image by taking "my mobile phone" as an example of the same account device, and it is easy to understand that other same account devices are similar to "my mobile phone", which will not be described here.

[0207] Taking "different account device" as an example, continuing to refer to FIG. 5C, the electronic device 100 can detect the operation (for example, click operation) of the user on the "different account device" option, in response to the operation, the electronic device 100 can display the user interface shown in FIG. 5J.

[0208] Referring to FIG. 5J, the user interface shown in FIG. 5J can be used to find the device logged in with a different account from the electronic device 100. There can be various ways to find, for example, input account-related information (for example, mobile phone number bound to Huawei account, etc.) in the input box shown in FIG. 5J, and for another example, the electronic device 100 can be triggered to discover the nearby device by operating (for example, click operation) on the "near field search discovery" option shown in FIG. 5J; and for another example, the electronic device 100 can be triggered to find the device of the corresponding contact in the contact list by operating (for example, click operation) on the "contact" option shown in FIG. 5J.

[0209] In one possible implementation, one or more same account devices shown in FIG. 5D can also be found by the method shown in FIG. 5J.

[0210] Taking the device of "different account device" as an example of the device of user Lily, the electronic device 100 can display one or more options (for example, the "Lily's mobile phone" option, the "Lily's tablet" option, the "Lily's car machine" option, and the like) corresponding to the device of user Lily in the user interface shown in FIG. 5K, and the one or more options can be used to trigger the setting of how the emotional cartoon image is displayed on the corresponding device. Wherein, the specific setting process is similar to the foregoing process of setting how the emotional cartoon image is displayed on the same account device, and the foregoing related content can be referred to, and details are not described herein again.

[0211] In a possible implementation, the electronic device 100 detects the setting operation of the user in the foregoing step S401, and specifically can include that the electronic device 100 detects that the user completes a series of setting operations such as FIGS. 5C-5I, and the series of setting operations can be used to set how the emotional cartoon image is displayed on the electronic device 200.

[0212] Referring to FIGS. 6A-6H, FIGS. 6A-6H are a series of exemplary user interfaces involved when the emotional cartoon image is displayed on different devices.

[0213] Taking the electronic device 100 (the local device) as a smart watch as an example, referring to FIG. 6A, the user interface shown in FIG. 6A is a user interface in which the emotional cartoon image is full-screen displayed on the electronic device 100.

[0214] Taking the electronic device 100 (the local device) as a smart watch as an example, referring to FIG. 6B, the user interface shown in FIG. 6B is a user interface in which the emotional cartoon image is non-full-screen displayed on the electronic device 100, for example, in the form of a service card displayed on the electronic device 100.

[0215] Taking the electronic device 200 (the remote device) as a folding screen mobile phone as an example, referring to FIG. 6C, the user interface shown in FIG. 6C is a user interface in which the emotional cartoon image is displayed on the outer screen of the folding screen of the electronic device 200, for example, the emotional cartoon image corresponding to the user (for example, user 1) of the electronic device 100 can be full-screen displayed on the outer screen of the folding screen of the electronic device 200. In the embodiment of the present application, the foregoing emotional cartoon image can also be displayed on the inner screen of the folding screen of the electronic device 200.

[0216] Taking the electronic device 200 (the remote device) as a mobile phone as an example, referring to FIG. 6D, the user interface shown in FIG. 6D is a user interface in which the emotional cartoon image is displayed on the minus one screen of the electronic device 200.

[0217] Taking the electronic device 200 (remote device) as a mobile phone as an example, referring to FIG. 6E, the user interface exemplarily shown in FIG. 6E is a user interface in which the emotional cartoon image is displayed when the electronic device 200 is in a screen-off state (i.e., the user interface involved when the emotional cartoon image is displayed on the screen-off display interface).

[0218] Taking the electronic device 200 (remote device) as a mobile phone as an example, referring to FIG. 6F, the user interface exemplarily shown in FIG. 6F is a user interface in which the emotional cartoon image is displayed in an application (for example, a sports health application) of the electronic device 200. As can be seen from FIG. 6F, the emotional cartoon image can be displayed in the form of a service card floating above the application interface, or displayed in the application interface. The display position of the emotional cartoon image on the application interface is not limited by the embodiments of the present application. In some examples, the electronic device 200 can also support the user to move the display position of the emotional cartoon image through a drag operation. In some examples, still referring to FIG. 6F, the user interface exemplarily shown in FIG. 6F is a user interface of a family space. In the case of being authorized, the electronic device 200 can support the user to view the emotional cartoon image corresponding to each user in the family space.

[0219] Taking the electronic device 200 (remote device) as a tablet as an example, referring to FIG. 6G, the user interface exemplarily shown in FIG. 6G is a user interface in which the emotional cartoon image is displayed on the desktop of the electronic device 200. As can be seen from FIG. 6G, the emotional cartoon image can be displayed in the form of a service card floating above the desktop, or displayed in the desktop.

[0220] Taking the electronic device 200 (remote device) as a car machine as an example, referring to FIG. 6H, the user interface exemplarily shown in FIG. 6H is a user interface in which the emotional cartoon image is displayed on the electronic device 200. The emotional cartoon image can be displayed in full screen on the electronic device 200, or can be displayed in non-full screen on the electronic device 200.

[0221] It should be noted that in the above FIGS. 6C-6H, the interface in which the emotional cartoon image is located can display information (for example, a user name) of the user corresponding to the emotional cartoon image, so that the user of the remote device can know which user the emotional cartoon image corresponds to. For example, if the emotional cartoon image corresponds to user 1, the interface in which the emotional cartoon image is located can display the information (for example, the user name "user 1") of the user. In some examples, the emotional cartoon image can be strongly related to a certain user, and after the user of the remote device sees the emotional cartoon image, the user can know which user the emotional cartoon image corresponds to. In this case, the interface in which the emotional cartoon image is located can also not display the information of the user corresponding to the emotional cartoon image. For example, in FIGS. 6C-6H, the text "user 1" can not be displayed.

[0222] In some examples, after the electronic device 100 updates the display of the emotional cartoon image, the remote device that is authorized to display the emotional cartoon image can also update the display of the emotional cartoon image in synchronization. In this way, the user of the remote device can be made to know the state of the user of the electronic device 100 in real time.

[0223] Embodiment three, personality type speculation, emotion prediction and adjustment

[0224] FIGS. 7A and 7B exemplarily show a specific flow of a personality type speculation, emotion prediction and adjustment method provided by an embodiment of the present application.

[0225] Referring to FIG. 7A, the method can be applied to the electronic device 100. The specific steps of the method are described in detail as follows:

[0226] S701, the electronic device 100 counts emotional data, wherein the emotional data includes emotion detection results of the user in a period of time and corresponding emotional associated factors.

[0227] The emotion detection results can include the belonging emotional category of the emotional state of the user, which can be obtained by classifying the emotion in N dimensions by using the aforementioned continuous emotion model (wherein N can be a positive integer greater than 1), or can be obtained by marking the emotion as discrete categories such as joy, anger, sorrow, grief, terror, surprise, etc. by using the aforementioned discrete emotion model, which is not limited in the embodiments of the present application.

[0228] Referring to FIG. 7B, the emotional associated factors can include but are not limited to environmental information (such as location, activity track, etc.), associated event information (such as social activities, daily events, etc.), and behavior performance information of the user (such as speech frequency, body movement, expression, etc.). In the embodiments of the present application, the associated event information can be obtained in various ways, for example, can be manually entered by the user, or can be obtained through device data (such as schedule data or application data), which is not limited in the embodiments of the present application.

[0229] It should be noted that the step S701 can be executed on the electronic device 100, or can be executed on a server, which is not limited in the embodiments of the present application.

[0230] After the electronic device 100 executes the step S701, the electronic device 100 can continue to execute the following steps S702-S703 to perform personality type speculation, or can continue to execute the following steps S704-S705 to perform emotion prediction and adjustment.

[0231] S702, the electronic device 100 speculates the personality type of the user based on the emotional data.

[0232] Specifically, referring to FIG. 7B, after a period of time of data is counted or a sufficient amount of data is obtained, the electronic device 100 can analyze the emotion-related factors included in the above-mentioned emotion data, and classify these emotion-related factors, for example, into a social class, an activity class, a hobby class, and the like. The social class can include, for example, a dinner party, a cocktail party, a dance party, and the like. The activity class can include, for example, a speech, a performance, and the like. The hobby class can include, for example, being alone, reading, writing, group sports, and the like. Further, the electronic device 100 can count the frequency of occurrence of the above-mentioned various classes of emotion-related factors, and can infer the personality type of the user based on the above-mentioned classified emotion-related factors and the frequency of occurrence thereof.

[0233] The personality type of the user can be, for example, an extrovert or an introvert. Alternatively, the personality type of the user can be, for example, various animal types (e.g., peacock type, panda type, tiger type, etc.).

[0234] For example, if the frequency of occurrence of the emotion-related factor "being alone" is high (i.e., the user is keen on being alone), it can be inferred that the personality type of the user is introverted.

[0235] For example, if the frequency of occurrence of the emotion-related factor "social class" is high (i.e., the user is keen on socializing), it can be inferred that the personality type of the user is extroverted.

[0236] For example, if the frequency of occurrence of the activity-related factor "speech" or "performance" is high (i.e., the user is keen on showing himself / herself), it can be inferred that the personality type of the user is peacock type.

[0237] It is easy to understand that, since the personality type of the user can periodically change over time or event triggers, the above-mentioned emotion data can be data counted over a period of time, so as to automatically infer the personality type of the user over the period of time.

[0238] In some examples, the inferred personality type of the user can be used to generate a subsequent personalized emotion regulation strategy.

[0239] S703, the electronic device 100 generates an emotion cartoon image corresponding to the personality type of the user based on the personality type of the user.

[0240] In the embodiments of the present application, the personality type of the user can be animalized, and the animalized personality type can be used as an optional input to generate an emotion cartoon image, thereby improving the interest.

[0241] For example, assuming that the personality type of the user is panda type, the above-mentioned emotion cartoon image can be a panda cartoon image capable of representing the emotion of the user.

[0242] It should be noted that the above steps S702-S703 can be executed on the electronic device 100 or on a server, and the embodiments of the present application do not limit this.

[0243] S704, after S701, the electronic device 100 detects that event 1 will occur after a preset time length, wherein event 1 is an event associated with the user appearing a specific emotion determined by the electronic device 100 based on the above-mentioned emotion data.

[0244] S705, the electronic device 100 determines the expected emotion of the user when event 1 occurs based on the above-mentioned emotion data, and provides the corresponding reminder and emotion regulation strategy to the user based on the expected emotion.

[0245] Since the above-mentioned emotion data includes the emotion detection result of the user in a period of time and its corresponding emotion associated factors (such as associated event information), the above-mentioned event 1 can be an event A in the associated event, so the emotion detection result of the user when the event A occurs can be found, and the emotion of the user at that time can be determined from the emotion detection result, which can be considered as the expected emotion of the user when the subsequent event 1 occurs again, thereby completing emotion prediction.

[0246] Exemplarily, assuming that the above-mentioned event 1 is a vehicle congestion event, when it is detected that event 1 will occur after a preset time length, the electronic device 100 can provide the corresponding reminder (such as “Please prepare for the upcoming vehicle congestion section”) to the user, and can also provide the emotion regulation strategy (such as providing some encouraging feedback to the user when the user produces negative emotion in the vehicle congestion event) to the user.

[0247] Exemplarily, assuming that the above-mentioned event 1 is a driver behavior event of searching for a parking lot, a gas station, a charging station, etc., when it is detected that event 1 will occur after a preset time length, the electronic device 100 can provide the corresponding reminder (such as “Please prepare for the upcoming destination, and search for a parking lot”) to the user, and can also provide the emotion regulation strategy (such as providing some encouraging feedback to the user when the user produces negative emotion in the event of searching for a parking lot, such as anxious and angry emotion when no parking lot is found nearby) to the user. In a possible implementation, the above-mentioned event 1 (such as searching for a parking lot, a gas station, a charging station, etc.) can be triggered after a preset time length by the vehicle state (such as vehicle speed, oil quantity, and electricity quantity).

[0248] For example, assuming that event 1 is that the user is attending a class, and the class is one that the user does not like, after detecting that event 1 will occur after a preset time period, the electronic device 100 can provide the user with a corresponding reminder (for example, "XX class is about to start, please prepare yourself") and can also provide the user with an emotion regulation strategy (for example, when the user starts learning a class that the user does not like, the user will have a negative emotion, and therefore, the user can be provided with some encouraging feedback). In one possible time manner, the above event 1 (for example, the user attending a class) can be determined by the user's schedule information.

[0249] In some examples, a mapping table can be pre-stored, which can reflect the mapping relationship between a scene and an emotion. One scene can correspond to a preset emotion. For example, the preset emotion corresponding to a party scene is a positive emotion. Therefore, in the party scene, if the emotion of the user detected by the electronic device 100 is not a positive emotion, the electronic device 100 can provide the user with a corresponding reminder and an emotion regulation strategy, so as to adjust the emotion of the user to a positive emotion.

[0250] It should be noted that the above-mentioned corresponding reminders and emotion regulation strategies provided to the user can be conveyed to the user through an emotion cartoon image (for example, conveyed to the user in the form of voice / motion / expression through an emotion cartoon image), or can not be conveyed to the user through an emotion cartoon image, but can be conveyed to the user in the form of traditional voice or text display. The embodiments of the present application are not limited in this regard.

[0251] It should be noted that the above-mentioned emotion prediction process can be performed on the electronic device 100 or on a server. The embodiments of the present application are not limited in this regard.

[0252] By implementing the method provided by the embodiments shown in FIGS. 7A and 7B, the personality type of the user can be predicted, and a corresponding emotion cartoon image can be generated based on the personality type of the user, thereby improving the interest. In addition, emotion prediction can be performed based on statistical emotion data, and corresponding reminders and emotion regulation strategies can be provided to the user, thereby effectively regulating the emotion of the user.

[0253] Embodiment Four, Generation of Personalized Emotion Regulation Strategies

[0254] FIG. 8A exemplarily shows a specific process of a personalized emotion regulation strategy generation method provided by an embodiment of the present application.

[0255] Referring to FIG. 8A, the method can be applied to the electronic device 100. The specific steps of the method are described in detail as follows:

[0256] S801, The electronic device 100 detects that the current emotion of the user is emotion A.

[0257] S802, the electronic device 100 detects a behavior of the user when the user is in the emotion A.

[0258] The behavior of the user in S802 can be a behavior of the user performing an emotion regulation intervention, or can be some behavior of the user in daily life. Referring to FIG. 8B, the behavior can include, but is not limited to, mindfulness exercises (such as breathing exercises, meditation exercises, etc.), various exercises (such as yoga, strength training, etc.), physical level stimulation (such as listening to music, eating delicious food, watching videos, playing games, massage, etc.), behavioral interventions (such as social confiding, sunbathing, sleeping, reading, etc.).

[0259] In some examples, the emotion regulation intervention described above can be an emotion regulation recommendation strategy from the system preset / default. When the electronic device 100 detects that the user is in the emotion A, the electronic device 100 can recommend the emotion regulation intervention to the user, for example, by recommending the emotion regulation intervention to the user through, for example, a pop-up window, a floating ball prompt, etc., so as to facilitate the user to quickly perform the content corresponding to the emotion regulation intervention. Alternatively, the user can also actively (not based on the behavior of emotion detection result) perform some behavior, which objectively has an impact on the user's emotion, that is, the same or similar to the emotion regulation intervention.

[0260] S803, the electronic device 100 detects that the emotion of the user after performing the corresponding behavior or action is emotion B, and evaluates the effectiveness of the emotion regulation intervention based on the change of the user from emotion A to emotion B.

[0261] For example, assuming that the emotion A is a negative emotion, and the emotion B after performing the emotion regulation intervention is still a negative emotion, it can be considered that the emotion regulation intervention is ineffective.

[0262] For example, assuming that the emotion A is a negative emotion, and the emotion B after performing the emotion regulation intervention is a neutral emotion or a positive emotion, it can be considered that the emotion regulation intervention is effective. It is easy to understand that, compared to the case where the emotion B is a neutral emotion, in the case where the emotion B is a positive emotion, the effectiveness of the emotion regulation intervention is higher.

[0263] It should be noted that the above only takes the case where the emotion adjustment intervention measure is considered effective in adjusting the user's emotion from a negative emotion to a neutral emotion or a positive emotion as an example, and should not be construed as a limitation of the present application, and in some other embodiments of the present application, the emotion can also be classified / sub-classified more finely, and the effectiveness of the emotion adjustment intervention measure can also be classified more finely. For example, the above emotion A and the above emotion B are both negative emotions, but the negative degree of the above emotion A is higher than that of the above emotion B, and in this case, the above emotion adjustment intervention measure can also be considered effective.

[0264] S804, the electronic device 100 generates a personalized emotion adjustment strategy based on the effectiveness of the above emotion adjustment intervention measure (optional, combined with the user personality type, the scenario factor, etc.).

[0265] In a possible implementation, the effectiveness of all emotion adjustment intervention measures used by the user can be sorted to determine the priority of each emotion adjustment intervention measure. It is easy to understand that the higher the effectiveness of the emotion adjustment intervention measure, the higher its priority in the generated personalized emotion adjustment strategy, and the user can be preferentially recommended the emotion adjustment intervention measure with higher priority in the case of needing to adjust the emotion.

[0266] In another possible implementation, the effectiveness of the emotion adjustment intervention measure and the user personality type can be comprehensively analyzed to generate a personalized emotion adjustment strategy. For example, the effectiveness of two emotion adjustment intervention measures is the same, but one of the emotion adjustment intervention measures is more in line with the user personality type, and therefore, in the generated personalized emotion adjustment strategy, the priority of this emotion adjustment intervention measure can be higher, and the user can be preferentially recommended this emotion adjustment intervention measure in the case of needing to adjust the emotion.

[0267] In another possible implementation, the effectiveness of the emotion adjustment intervention measure and the scenario factor (such as an event, a time, a place, etc.) can be comprehensively analyzed to generate a personalized emotion adjustment strategy. For example, the effectiveness of two emotion adjustment intervention measures is the same, but one of the emotion adjustment intervention measures is more suitable for the current scenario of the user, and therefore, in the generated personalized emotion adjustment strategy, the priority of this emotion adjustment intervention measure can be higher, and the user can be preferentially recommended this emotion adjustment intervention measure in the case of needing to adjust the emotion. Illustratively, in the driving scenario, the two emotion adjustment intervention measures are watching a video and listening to music, and their effectiveness is the same, and it is easy to understand that listening to music is more suitable for the driving scenario than watching a video, and therefore, the user can be preferentially recommended the emotion adjustment intervention measure of listening to music in the case of needing to adjust the emotion.

[0268] In another possible implementation, referring to FIG. 8B, the effectiveness of the emotional regulation intervention measures, the user personality type, and the scenario factors can be comprehensively analyzed to generate a personalized emotional regulation strategy. For example, weights can be respectively assigned to the effectiveness of the emotional regulation intervention measures, the user personality type, and the scenario factors, so as to further determine a total score of each emotional regulation intervention measure. It can be easily understood that the higher the total score of the emotional regulation intervention measure, the higher the priority of the emotional regulation intervention measure in the generated personalized emotional regulation strategy. When the user needs to perform emotional regulation, the user can be preferentially recommended the emotional regulation intervention measure with a higher priority.

[0269] In some examples, a more refined personalized emotional regulation strategy can also be generated. For example, the field can be subdivided. For example, the same emotional regulation can be performed through exercise, but different emotional states can be suitable for different exercise intensities. The user can be prompted to perform which exercise intensity based on the current emotional state of the user (for example, high-intensity exercise is not suitable when the user is in a bad mood). For another example, the same emotional regulation can be performed through music, but different emotional states can be suitable for different music types. The user can be prompted to perform which music type based on the current emotional state of the user (for example, light music is suitable when the user is in a bad mood).

[0270] In the embodiment of the present application, after the personalized emotional regulation strategy is generated, referring to FIG. 8B, the optimization system preset / default emotional regulation recommendation strategy can be fed back to provide the user with a more effective emotional regulation strategy.

[0271] By implementing the method provided in the embodiment shown in FIG. 8A, the effectiveness of the emotional regulation intervention measures, the user personality type, and the scenario factors can be comprehensively analyzed to generate a personalized emotional regulation strategy. In this way, the effectiveness of the emotional regulation strategy can be guaranteed, and the emotional regulation strategy can be consistent with the user personality type and the current scenario of the user (that is, a suitable emotional regulation strategy is recommended at a suitable time and in a suitable place).

[0272] Embodiment Five: Emotional regulation through interaction with an emotional cartoon image

[0273] In the embodiment of the present application, the electronic device 100 can support the user to interact with the emotional cartoon image, so that different feedback can be given to the user based on different emotional states. In this way, the user's emotion can be effectively regulated, and the user experience can be improved.

[0274] In the embodiment of the present application, referring to FIG. 8C, a plurality of ways can be used to regulate the user's emotion, such as playing a video / animation (that is, video / animation feedback), playing audio (that is, audio feedback), talking and confessing (that is, voice or text communication), mindfulness practice (or mindfulness guidance), a small game, and the like.

[0275] The above-mentioned several modes will be exemplarily introduced in combination with a series of user interfaces.

[0276] 1. Playing a video / animation

[0277] In the embodiments of the present application, the electronic device 100 can adjust the user's mood by playing a video / animation. Different mood states can be given different types (e.g., encouraging, comforting, etc.) of video / animation feedback.

[0278] Exemplarily, referring to FIG. 9A, when it is detected that the user is currently in an XX mood state, the electronic device 100 can display the prompt information exemplarily shown in FIG. 9A to prompt whether the user needs to adjust the mood by playing a video / animation. If the user agrees (e.g., the user clicks the "Yes" option shown in FIG. 9A), the electronic device 100 can play a corresponding video / animation based on the user's current mood state. For example, when the user feels very accomplished after completing a difficult task, the user is in a positive mood state, and then the user can be given an encouraging feedback, such as playing an encouraging video / animation (e.g., a video / animation of an emotional cartoon figure giving a thumbs-up) exemplarily shown in FIG. 9B. For another example, when the user is in a sad mood state, the user can be given a comforting feedback, such as playing a comforting video / animation (e.g., a video / animation of an emotional cartoon figure giving a hug) exemplarily shown in FIG. 9C. Or, for another example, as shown in (2)-(8) of FIG. 17G or (2)-(6) of FIG. 17H, a comforting video / animation can be played.

[0279] In some examples, before playing the video / animation, the electronic device 100 can also not display the prompt information shown in FIG. 9A, but after detecting that the user operates (e.g., a click operation) on the emotional cartoon figure displayed on the electronic device 100, a corresponding video / animation can be played based on the user's current mood state. Or, a video / animation can be directly played when it is detected that the user is currently in a certain preset mood state, to actively adjust the user's mood. For example, when it is detected that the user is currently in a negative mood state, a corresponding video / animation can be directly played based on the negative mood state. Or, a video / animation can be directly played with the prior authorization of the user, to actively adjust the user's mood. And so on.

[0280] 2. Playing audio

[0281] In the embodiments of the present application, the electronic device 100 can adjust the user's mood by playing audio. Different mood states can be given different types (e.g., a hit song, R&B, light music, etc.) of audio feedback.

[0282] Exemplarily, referring to FIG. 10A, when detecting that the user is currently in the XX emotional state, the electronic device 100 can display the prompt information exemplarily shown in FIG. 10A to prompt the user whether to need to adjust the emotion by playing music, if the user agrees (for example, the user clicks the "Yes" option shown in FIG. 10A), the electronic device 100 can play the corresponding audio based on the current emotional state of the user. For example, when the user is in a positive emotional state, a hit song can be played; for another example, when the user is in a neutral emotional state, R&B can be played; for another example, when the user is in a negative or high-stress emotional state, light music can be played; and the like.

[0283] Optionally, the electronic device 100 can display the emotional cartoon character singing in the process of playing music, as exemplarily shown in FIG. 10B.

[0284] In some examples, the electronic device 100 can also not display the prompt information shown in the foregoing FIG. 10A before playing the audio. For example, in the case of obtaining the user's authorization in advance, the audio can be directly played to actively adjust the user's emotion; for another example, in a preset scene (for example, a home scene, a vehicle-mounted scene, etc.), the audio can be directly played to actively adjust the user's emotion; and the like.

[0285] 3. Talk confession (i.e., voice or text communication)

[0286] In the embodiments of the present application, the electronic device 100 can adjust the user's emotion by talk confession, in which case the electronic device 100 can assume the role of an artificial intelligence chat robot to guide the user to talk confession.

[0287] Exemplarily, referring to FIG. 11A, when detecting that the user is currently in the XX emotional state, the electronic device 100 can display the prompt information exemplarily shown in FIG. 11A to prompt the user whether to need to adjust the emotion by talk confession, if the user agrees (for example, the user clicks the "Yes" option shown in FIG. 11A), the electronic device 100 can assume the role of an artificial intelligence chat robot to guide the user to talk confession. For example, the electronic device 100 can display a cartoon character in a chatting state exemplarily shown in FIG. 11B, and the electronic device 100 can support the user to communicate with it by voice or text.

[0288] In some examples, the content guiding the user to talk confession can be related to the event associated with the user's current emotion, for example, the user has a negative emotion because of work, then the content guiding the user to talk confession output by the electronic device 100 can be work-related content (for example, "Wasn't today's work not smooth? Tell me about it").

[0289] In some examples, the electronic device 100 can also not display the prompt information shown in the foregoing FIG. 11A before the talk confession. For example, in the case of obtaining the user's authorization in advance, the electronic device 100 can assume the form of an artificial intelligence chat robot to guide the user to talk confession and actively adjust the user's emotion; for example, in the preset scene (such as a home scene, etc.), the electronic device 100 can assume the form of an artificial intelligence chat robot to guide the user to talk confession and actively adjust the user's emotion; and the like.

[0290] 4. Mindfulness practice (or mindfulness guidance)

[0291] In the embodiments of the present application, the electronic device 100 can improve the user's concentration through mindfulness practice, so as to achieve the purpose of adjusting the user's emotion.

[0292] Exemplarily, referring to FIG. 12A, when detecting that the user is currently in an XX emotional state, the electronic device 100 can display the prompt information exemplarily shown in FIG. 12A to prompt whether the user needs to adjust the emotion through mindfulness practice. If the user agrees (for example, the user clicks the “Yes” option shown in FIG. 12A), the electronic device 100 can display the user interface exemplarily shown in FIG. 12B, which can include one or more options (such as “breathing training” option, “meditation training” option). The one or more options can be used to trigger the electronic device 100 to play a video / animation for guiding the user to perform mindfulness practice.

[0293] Exemplarily, the electronic device 100 can detect the user's operation (such as click operation) on the “breathing training” option shown in FIG. 12B. In response to the operation, the electronic device 100 can play a video / animation for guiding the user to perform breathing training, such as exemplarily shown in FIG. 12C.

[0294] Exemplarily, the electronic device 100 can detect the user's operation (such as click operation) on the “meditation training” option shown in FIG. 12B. In response to the operation, the electronic device 100 can play a video / animation for guiding the user to perform meditation training, such as exemplarily shown in FIG. 12D.

[0295] In some examples, the electronic device 100 can also not display the prompt information shown in the foregoing FIG. 12A before the mindfulness practice. For example, in the case of obtaining the user's authorization in advance, the electronic device 100 can directly play a video / animation for guiding the user to perform mindfulness practice and actively adjust the user's emotion; for example, in the preset scene (such as a home scene, etc.), the electronic device 100 can directly play a video / animation for guiding the user to perform mindfulness practice and actively adjust the user's emotion; and the like.

[0296] 5. Small game

[0297] In the embodiment of the present application, the electronic device 100 can divert the user's attention through the mini-game, so as to achieve the purpose of regulating the user's mood.

[0298] For example, referring to FIG. 13A, when it is detected that the user is currently in the XX mood state, the electronic device 100 can display the prompt information as exemplarily shown in FIG. 13A to prompt whether the user needs to regulate the mood through the mini-game. If the user agrees (for example, the user clicks the “Yes” option shown in FIG. 13A), the electronic device 100 can display the user interface as exemplarily shown in FIG. 13B, which can include one or more mini-game options (for example, the “Wooden Fish” mini-game option, the “Buddhist Bead” mini-game option). The one or more mini-game options can be used to trigger the electronic device 100 to display the corresponding mini-game interface.

[0299] For example, the electronic device 100 can detect the user's operation (for example, a click operation) on the “Wooden Fish” mini-game option shown in FIG. 13B. In response to the operation, the electronic device 100 can display the “Wooden Fish” mini-game interface as exemplarily shown in FIG. 13C. The electronic device 100 can support the user to regulate his mood by playing the “Wooden Fish” mini-game.

[0300] For example, the electronic device 100 can detect the user's operation (for example, a click operation) on the “Wooden Fish” mini-game option shown in FIG. 13B. In response to the operation, the electronic device 100 can display the “Wooden Fish” mini-game interface as exemplarily shown in FIG. 13C. The electronic device 100 can support the user to regulate his mood by playing the “Wooden Fish” mini-game.

[0301] In some examples, the electronic device 100 can also not display the prompt information as shown in FIG. 13A before running the mini-game. For example, in the case of obtaining the user's authorization in advance, the electronic device 100 can directly display the recommended mini-game to the user and actively recommend the corresponding mini-game to the user to regulate the user's mood; for example, in a preset scene (for example, a home scene, etc.), the electronic device 100 can directly display the recommended mini-game to the user and actively recommend the corresponding mini-game to the user to regulate the user's mood; and the like.

[0302] It should be noted that, since in the embodiment of the present application, the user's mood can be regulated in multiple ways, there are multiple methods for determining which way to use to regulate the user's mood, and the present embodiment does not limit this.

[0303] In a possible implementation, which mode is used to regulate the user's emotion can be determined based on the personalized emotion regulation strategy generated in the foregoing embodiment four, for example, the mode with the highest priority can be used to regulate the user's emotion.

[0304] In another possible implementation, which mode is used to regulate the user's emotion can be determined based on the user's selection. Illustratively, referring to FIG. 14, the user can actively trigger the electronic device 100 to display the user interface shown in the example of FIG. 14, which can include one or more options corresponding to the emotion regulation modes (for example, a "video / animation" option, an "audio" option, a "voice / text communication" option, a "mindfulness practice" option, and a "mini-game" option), and the electronic device 100 can support the user to operate (for example, click) any one of the options, after the operation, the electronic device 100 can give the user corresponding feedback to regulate the user's emotion. For example, the electronic device 100 can detect the user's operation on the "video / animation" option, and in response to the operation, the electronic device 100 can play corresponding video / animation based on the user's current emotional state; for another example, the electronic device 100 can detect the user's operation on the "audio" option, and in response to the operation, the electronic device 100 can play corresponding audio based on the user's current emotional state; for another example, the electronic device 100 can detect the user's operation on the "voice / text communication" option, and in response to the operation, the electronic device 100 can assume the form of an artificial intelligence chat robot to guide the user to talk and complain; for another example, the electronic device 100 can detect the user's operation on the "mindfulness practice" option, and in response to the operation, the electronic device 100 can display the user interface shown in FIG. 12B, and further, the user can operate the "breathing training" option or the "meditation training" option in the user interface to trigger the electronic device 100 to play video / animation for guiding the user to practice mindfulness; for another example, the electronic device 100 can detect the user's operation on the "mini-game" option, and in response to the operation, the electronic device 100 can display the user interface shown in FIG. 13B, and further, the user can operate the "wooden fish knocking" mini-game option or the "prayer beads" mini-game option in the user interface to trigger the electronic device 100 to display the corresponding mini-game interface.

[0305] It should be noted that in the above process of introducing the mood regulation modes of playing video / animation, playing audio, talking and confiding (i.e. voice or text communication), mindfulness exercise (or mindfulness guide), and small game, the mood of the user of the electronic device 100 is taken as an example. In some examples, since the mood cartoon image of other users (e.g. family members, friends, etc.) can also be displayed on the electronic device 100, the electronic device 100 can also support the local user to regulate the mood of the other users by operating the mood cartoon image of the other users displayed on the electronic device 100, and the mood regulation mode used can also be one of the above-mentioned modes. The following will be introduced exemplarily.

[0306] Referring to FIG. 15A, assuming that the mood cartoon image displayed by the electronic device 100 is the mood cartoon image of the user of the electronic device 200, in one possible implementation, the electronic device 100 can display an option (e.g. the heart-shaped option exemplarily shown in FIG. 15A) for triggering the mood regulation of the user of the electronic device 200, the electronic device 100 can detect the operation (e.g. click operation) of the user on the option, and in response to the operation, the electronic device 100 can display the user interface exemplarily shown in FIG. 14, which can include one or more mood regulation mode corresponding options (e.g. “video / animation” option, “audio” option, “voice / text communication” option, “mindfulness exercise” option, “small game” option), which can be used by the user to select which mode to use to regulate the mood of the user of the electronic device 200.

[0307] Taking video / animation as an example, the electronic device 100 can detect the operation of the user on the "video / animation" option shown in FIG. 14, and in response to the operation, the electronic device 100 can display the user interface shown in FIG. 15B, which can include one or more video / animation corresponding options (e.g., a "like" option, a "hug" option, a "head touch" option, a "pat" option, a "scold" option), which can trigger the emotional cartoon character to perform a corresponding action, which can be used to indicate the action made by the user of the electronic device 100 (e.g., user XXX) to the user of the electronic device 200, so as to adjust the emotion of the user of the electronic device 200 through the action. For example, the electronic device 100 can detect the operation (e.g., a click operation) of the user on the "like" option shown in FIG. 15B, and in response to the operation, the electronic device 100 can send a feedback result to the electronic device 200, which can include information that the emotional cartoon character performs a like action, and after the electronic device 200 receives the feedback result, the electronic device 200 can display the video / animation of the emotional cartoon character giving a thumbs-up shown in FIG. 15C based on the feedback result. Alternatively, referring to FIG. 15C, the electronic device 200 can also output related prompt information (e.g., a voice prompt "user XXX likes you") in the form of voice, text, etc., to prompt the user who wants to adjust the emotion of the user of the electronic device 200 through the video / animation feedback.

[0308] Taking audio as an example, the electronic device 100 can detect the operation of the user on the "audio" option shown in FIG. 14, and in response to the operation, the electronic device 100 can send a feedback result to the electronic device 200, which can be used to indicate that the user of the electronic device 100 (e.g., user XXX) wants to adjust the emotion of the user of the electronic device 200 by listening to music, and after the electronic device 200 receives the feedback result, the electronic device 200 can output the prompt information shown in FIG. 15D to prompt the user to adjust the emotion by listening to music, and if the user agrees (e.g., the user clicks the "yes" option shown in FIG. 15D), the electronic device 200 can give the user corresponding audio feedback based on the emotional state of the user, and the specific process can refer to the foregoing description of the emotional adjustment mode of playing audio, which will not be described here.

[0309] For example, the electronic device 100 can detect the operation of the user on the "voice / text communication" option shown in FIG. 14, and in response to the operation, the electronic device 100 can send a feedback result to the electronic device 200, which can be used to indicate that the user (e.g., user XXX) of the electronic device 100 wants to adjust the mood of the user of the electronic device 200 through conversation confession. After the electronic device 200 receives the feedback result, it can output the prompt information shown in FIG. 15E to prompt the user to adjust the mood through conversation confession. If the user agrees (e.g., the user clicks the "Yes" option shown in FIG. 15E), the electronic device 200 can assume the role of an artificial intelligence chat robot to guide the user to have a conversation confession. The specific process can refer to the foregoing description of the conversation confession mood adjustment method, and will not be described here.

[0310] For example, the electronic device 100 can detect the operation of the user on the "mindfulness exercise" option shown in FIG. 14, and in response to the operation, the electronic device 100 can send a feedback result to the electronic device 200, which can be used to indicate that the user (e.g., user XXX) of the electronic device 100 wants to adjust the mood of the user of the electronic device 200 through mindfulness exercise. After the electronic device 200 receives the feedback result, it can output the prompt information shown in FIG. 15F to prompt the user to adjust the mood through mindfulness exercise. If the user agrees (e.g., the user clicks the "Yes" option shown in FIG. 15F), the electronic device 200 can provide one or more mindfulness exercise items to the user to adjust the mood through mindfulness exercise. The specific process can refer to the foregoing description of the mindfulness exercise mood adjustment method, and will not be described here.

[0311] For example, the electronic device 100 can detect the operation of the user on the "mindfulness exercise" option shown in FIG. 14, and in response to the operation, the electronic device 100 can send a feedback result to the electronic device 200, which can be used to indicate that the user (e.g., user XXX) of the electronic device 100 wants to adjust the mood of the user of the electronic device 200 through mindfulness exercise. After the electronic device 200 receives the feedback result, it can output the prompt information shown in FIG. 15F to prompt the user to adjust the mood through mindfulness exercise. If the user agrees (e.g., the user clicks the "Yes" option shown in FIG. 15F), the electronic device 200 can provide one or more mindfulness exercise items to the user to adjust the mood through mindfulness exercise. The specific process can refer to the foregoing description of the mindfulness exercise mood adjustment method, and will not be described here.

[0312] It should be noted that the above is only an example of taking the electronic device 100 as the local device and the electronic device 200 as the remote device to introduce that the electronic device 100 adjusts the emotion of the other user by the user operating the emotional cartoon image of the other user displayed on the electronic device 100. It can be understood that for the user of the electronic device 200, the electronic device 200 is the local device, and the electronic device 100 is the remote device. In this case, the emotional cartoon image of the user of the electronic device 100 can also be displayed on the electronic device 200, that is, the emotional cartoon image generated by the electronic device 100 can be sent to the electronic device 200 for display. Further, the electronic device 200 can also support the user to trigger the electronic device 200 to send the feedback result to the electronic device 100 by operating the emotional cartoon image of the user of the electronic device 100 displayed on the electronic device 200 to adjust the emotion of the user of the electronic device 100. The specific process is similar to the specific process of the user operating the emotional cartoon image of the other user displayed on the electronic device 100 to adjust the emotion of the other user, and the foregoing related description can be referred to.

[0313] It should be noted that in the above embodiment five, the emotion regulation is performed by the user interacting with the emotional cartoon image, but is not limited thereto. In some examples, the process of emotion regulation can also be irrelevant to the emotional cartoon image. For example, the electronic device 100 can detect the emotion of the user in real time, and provide the user with corresponding reminders and emotion regulation strategies based on the emotion detection result.

[0314] FIG. 16 exemplarily shows a schematic diagram of an interaction method provided by an embodiment of the present application.

[0315] As shown in FIG. 16, the method can include:

[0316] S1601, the electronic device 100 detects the emotion of the user 1. Specifically, the electronic device 100 can collect the physiological data of the user 1, and perform emotion detection on the user 1 based on the physiological data of the user 1 to obtain an emotion detection result, and generate an emotional cartoon image in a first emotion based on the emotion detection result or based on the emotion detection result and the personalized information of the user, or can be called a first virtual image in a first emotion.

[0317] The step of generating the emotional cartoon image can refer to the description in the embodiments shown in FIG. 2A, FIG. 2B and FIG. 2C. In a possible implementation, the first virtual image in the first emotion can be generated based on first data, which is physiological data of the first user for determining the emotion; or, the first virtual image in the first emotion can be generated based on the first data and second data, the second data including personalized information of the user, for example, including one or more of the following: environmental information of the user, sports health data, event information associated with generating the first emotion, and personality type of the user.

[0318] That is to say, the first virtual image can be generated based on only the emotion of the user, or can be generated based on multiple factors (for example, the emotion of the user, environmental information of the user, sports health data, event information associated with generating the current emotion, and personality type of the user).

[0319] In a possible implementation, after the step S1601, the electronic device 100 can display the generated emotional cartoon image in the electronic device 100 based on some trigger conditions. For example, when the electronic device 100 is bright, or at a specific time, a specific event, clicking a specific entity or virtual button, etc., the emotional cartoon image can be triggered to be displayed. The trigger conditions can refer to the trigger conditions in S1602, for example, including: a specific user action, a specific event, a specific wake-up word, a change in the emotion detection result satisfying a first preset condition, a change in the environmental information of the user satisfying a second preset condition, or a change in the sports health data of the user satisfying a third preset condition.

[0320] The electronic device 100 can periodically execute the above step S1601, or can trigger the execution of the above step S1601 based on a change in physiological data or emotion detection result, or can trigger the execution of the above step S1601 based on a change in some personalized information of the user (for example, environmental information, sports health status or associated events, etc.).

[0321] S1602, the electronic device 100 sends a message 1 to the electronic device 200, the message 1 being used to instruct the electronic device 200 to display the first virtual image in the first emotion. The first emotion is the emotion of the user 1 detected by the electronic device 100 in S1601.

[0322] The electronic device 200 and the electronic device 100 can be in the same home space or group, or the electronic device 200 can be a device of a contact of the electronic device 100. The relationship and communication mode between the electronic device 100 and the electronic device 200 can refer to the description in the embodiments shown in FIG. 4A and FIG. 4B.

[0323] S1602 can be executed when a trigger condition is met. The trigger condition can be set by the user, for example, the user can set the above-mentioned emotional cartoon image to be displayed on a certain device or devices, or can be displayed on a certain interface of the device, or can be displayed in a certain time period, etc. The specific setting process can refer to the description in the foregoing embodiments shown in FIGS. 4A, 4B, 5A-5K.

[0324] In the embodiments of the present application, the trigger condition can include but is not limited to one or more of the following: a specific user action, a specific event, a specific wake-up word, a change in the emotional detection result meeting a first preset condition, a change in the environment information of the user meeting a second preset condition, or a change in the motion health data of the user meeting a third preset condition.

[0325] The specific user action can include but is not limited to one or more of the following: a user shaking the arm wearing the electronic device 100, a user clicking or sliding on the screen, a user pressing the crown, a user lifting the wrist, etc.

[0326] For example, taking the user shaking the arm wearing the electronic device 100 as an example, the user can wear the electronic device 100 on one of his arms, after wearing, the user can shake the arm, after the electronic device 100 detects the user's shaking action (such as shaking the arm for two or three times), the electronic device 100 can display the shaking head and shaking brain emotional cartoon image, and send the above-mentioned message 1 to the electronic device 200.

[0327] The specific event can include but is not limited to one or more of the following: a specific scene, a specific time point, or a specific time period.

[0328] For a specific scenario, in embodiments of the present application, the electronic device 100 can display an emotional cartoon image associated with the scenario and send the above message 1 to the electronic device 200 when detecting that the user is in a specific scenario. The specific scenario (or activity) can include one or more of the following: work, study, class, exercise, travel, listen to music, eat, drink milk tea, play games, drive, take a car, sports, sleep. In embodiments of the present application, the electronic device 100 can identify the scenario by one or more of the following: sensor identification, application information, preset mode. For example, in the exercise scenario, the electronic device 100 can identify whether the user is exercising by using an acceleration sensor, a gyroscope sensor, etc., or can identify whether the user is exercising according to exercise data in a sports health application, or can identify whether the user is exercising according to a preset time period. For another example, in the sleep scenario, the electronic device 100 can identify whether the user is sleeping according to sleep data in a sports health application, or can identify whether the user is sleeping according to a preset time period. For another example, in the dining scenario, the electronic device 100 can identify whether the user is dining according to location data in a navigation application, or can identify whether the user is dining according to a preset time period. Further, in some possible implementations, the electronic device 100 can determine the emotional cartoon image according to data collected by a sensor. It can be understood that the data collected by the sensor is associated with the specific scenario in which the user is located. That is, after determining the specific scenario in which the user is located, the emotional cartoon image can be determined according to the data collected by the sensor and associated with the specific scenario.

[0329] For a specific time point or a specific time period, the electronic device 100 can display an emotional cartoon image associated with the time (for example, display an emotional cartoon image with a current time background) and send the above message 1 to the electronic device 200 when detecting that the current time point reaches the specific time point, or detecting that the current time point is within the specific time period. In some examples, when detecting that the current time point is within the specific time period, the electronic device 100 can display the emotional cartoon image after detecting a specific user operation and send the above message 1 to the electronic device 200. For example, when detecting a wrist-lifting operation of the user, the electronic device 100 displays the emotional cartoon image after the screen is turned on and sends the above message 1 to the electronic device 200.

[0330] The specific wake-up word can be a fixed wake-up word preset by the system of the electronic device 100, or a custom wake-up word (or preset wake-up word) set by the user, and the embodiments of the present application do not limit this. For example, the user can input the specific wake-up word to the electronic device 100 in the form of voice, and after the electronic device 100 detects the specific wake-up word input by the user, the electronic device 100 can display the emotional cartoon image and send the message 1 to the electronic device 200.

[0331] In a possible implementation, the trigger condition can be met, and specifically, the electronic device 100 can detect a setting operation of the user, and the setting operation can be used to authorize the emotional cartoon image to be displayed on the electronic device 200 in a certain time period. In this case, the electronic device 200 can display the emotional cartoon image when the start time point of the authorized time period is reached.

[0332] In another possible implementation, the trigger condition can be met, and specifically, the electronic device 100 can detect that the change of the emotion detection result of the user 1 meets a first preset condition. For example, the first preset condition can mean that the type of the emotion of the user 1 has changed. For example, the type of the emotion includes positive emotion (for example, happy), neutral emotion (for example, calm), and negative emotion (for example, unhappy). The first preset condition can be met, and specifically, the emotion of the user can change from positive emotion to neutral emotion / negative emotion, or the emotion of the user can change from neutral emotion to positive emotion / negative emotion, or the emotion of the user can change from negative emotion to neutral emotion / positive emotion.

[0333] In another possible implementation, the trigger condition can be met, and specifically, the electronic device 100 can detect that the change of the environment information in which the user 1 is located meets a second preset condition. For example, the second preset condition can include one or more of the following: the weather of the current location of the user changes, or the location of the user changes. The second preset condition can be met, and specifically, the weather of the current location of the user can change from sunny / cloudy / overcast to rain / snow, or the weather of the current location of the user can change from rain / snow to sunny / cloudy / overcast; and / or the location of the user can change from location 1 to location 2 (for example, from the living room at home to the bedroom at home, from home to the company, from one city to another city, and the like).

[0334] In another possible implementation, the trigger condition is satisfied, and specifically can include that the electronic device 100 detects that the change in the motion health data of the user 1 satisfies a third preset condition, where the third preset condition can include one or more of the following: the sleep data of the user 1 has changed, the motion data of the user 1 has changed, and the health data of the user 1 has changed. The third preset condition is satisfied, and specifically can include that the user 1 changes from the falling asleep state to the getting up state, or the user 1 changes from the getting up state to the waiting to fall asleep state; and / or, the user 1 changes from the pre-motion state to the in-motion state, or the user 1 changes from the in-motion state to the post-motion state; and / or, the data such as the heart rate, body temperature, blood pressure, blood sugar, etc. of the user 1 changes by a value greater than a certain preset threshold.

[0335] S1603, the electronic device 200 displays an interface 1 based on the message 1, where the interface 1 includes the above-mentioned emotion cartoon image in the first emotion; then the electronic device 200 detects an operation 1 of the user 2 on the above-mentioned emotion cartoon image, and in response to the operation 1, the electronic device 200 sends a message 2 to the electronic device 100.

[0336] Wherein, the process of displaying the above-mentioned emotion cartoon image by the electronic device 200 can refer to the description in the embodiments shown in FIGS. 6A-6H. For example, the above-mentioned emotion cartoon image can be displayed in full screen, or the above-mentioned emotion cartoon image can be displayed in non-full screen, for example, in the form of a card.

[0337] Wherein, the above-mentioned user 2 can be a certain member in the home space of the above-mentioned user 1, or can be a contact of the user 1.

[0338] Wherein, the above-mentioned operation 1 can be an operation on the above-mentioned emotion cartoon image; or, the above-mentioned operation 1 can be an operation on a function control included in the interface displaying the above-mentioned emotion cartoon image.

[0339] Wherein, in the case that the above-mentioned operation 1 is an operation on the above-mentioned emotion cartoon image, the above-mentioned operation 1 can be, for example, a touch operation, a pat operation, etc. For example, the user 2 can operate on the cartoon image as shown in FIGS. 6C-6H. Alternatively, in response to the operation of the user 2 on the cartoon image as shown in FIGS. 6C-6H, the electronic device 200 can display a plurality of options as shown in FIG. 14. Further, the user 2 can operate on the plurality of options, and in response to the further operation of the user 2, the electronic device 200 sends the message 2 to the electronic device 100.

[0340] In the case that the operation 1 is an operation on an option of a function control included in an interface for displaying the emotional cartoon image, the operation 1 can be an operation of a user clicking on, for example, the heart-shaped option shown in FIG. 15A; or, after the user clicks on, for example, the heart-shaped option shown in FIG. 15A, triggering the display of a plurality of options of the emotional regulation service shown in FIG. 14, the user clicks on, for example, the video / animation option, triggering the display of the options of the video / animation feedback shown in FIG. 15B, and the user clicks on one of the options; or, after the user clicks on, for example, the heart-shaped option shown in FIG. 15A, triggering the display of a plurality of options (such as patting, touching, audio, voice / text communication, mindfulness practice, small game, etc.), the user clicks on one of the options.

[0341] In a possible implementation, the electronic device 200 displays the emotional cartoon image based on the message 1, which can specifically include that the electronic device 200 directly displays the emotional cartoon image based on the message 1 after receiving the message 1, or the electronic device 200 does not display the emotional cartoon image after receiving the message 1, but displays the emotional cartoon image after detecting an operation of the user viewing the emotional cartoon image.

[0342] In a possible implementation, before the electronic device 200 displays the interface 1 in S1603, the method can further include that the electronic device 100 detects an operation of the user granting the electronic device 200 the permission to display the first virtual image. In this way, only the electronic device 100 can display the first virtual image, and other electronic devices (i.e., remote devices) cannot display the first virtual image, in the case that the user 1 authorizes on the electronic device 100.

[0343] In a possible implementation, the permission to display the virtual image can include a permission to display the virtual image in a specific interface, and / or a permission to display the virtual image in a first time period. The specific interface can include one or more of a negative screen interface, a lock screen interface, a desktop, an off-screen display interface, and an application interface.

[0344] S1604, the electronic device 100 receives the message 2 and performs a corresponding function based on the message 2.

[0345] The corresponding function includes but is not limited to one or more of the following functions: outputting an interface including a video / animation, outputting an audio, running an artificial intelligence chat, running a mindfulness practice (or mindfulness guide), and running a small game. For details of the functions for emotional regulation, see the description of Embodiment V.

[0346] The electronic device 100 can directly perform the corresponding function based on the message 2, for example, directly display the screen as shown in FIG. 9B, FIG. 9C, or directly play the corresponding music as shown in FIG. 10B, or directly start the communication with the user 1 as shown in FIG. 11B, or start to guide the user as shown in FIG. 12B-FIG. 12D, FIG. 13B-FIG. 13D. Among them, the electronic device 100 can also display information related to the user 2 when performing the corresponding function; for example, the electronic device 100 can display text information in addition to the animation shown in FIG. 9B: User 2 likes you; or the virtual image displayed in the electronic device 100 can be the virtual image of the user 2, thereby directly indicating that the user 2 is the feedback to the user 1, such as like, hug, etc.

[0347] Alternatively, the electronic device 100 can first prompt the user 1 whether to perform the corresponding function based on the message 2, and then perform the corresponding function after receiving the confirmation of the user 1. Similar to FIG. 9A, FIG. 10A, FIG. 11A, FIG. 12A or FIG. 13A, the electronic device 100 can first prompt the user 1, and different from FIG. 9A, FIG. 10A, FIG. 11A, FIG. 12A or FIG. 13A, the electronic device 100 can display that the prompt is based on the feedback of the user 2 when displaying the prompt in response to the message 2. For example: similar to FIG. 9A, the electronic device 100 can display: Received the concern of the user 2, do you want to play the video / animation? Or, the user 2 likes you XXX video / audio, do you want to watch?

[0348] Among them, the above-mentioned video / animation can be a video / animation for adjusting the emotion of the user 1.

[0349] In some examples, the above-mentioned video / animation or the interface including the above-mentioned video / animation can display the above-mentioned emotional cartoon character in a certain state, which can include but is not limited to: performing a certain action, being performed a certain action, being in a certain posture. The state can correspond to the above-mentioned operation 1. Among them, when the first emotion of the user 1 is a negative emotion, the operation 1 of the user 2 can be an operation for soothing the user 1; and the interface including the above-mentioned video / animation can be an interface for soothing the emotional cartoon character. For example, the above-mentioned operation 1 is a like operation performed on the above-mentioned emotional cartoon character, and the above-mentioned emotional cartoon character performing the like action can be displayed in the above-mentioned video / animation; for another example, the above-mentioned operation 1 is a touch operation (or a hug operation) performed on the above-mentioned emotional cartoon character, and the above-mentioned emotional cartoon character being touched (or hugged) can be displayed in the above-mentioned video / animation; for another example, the above-mentioned emotional cartoon character is in a negative emotion, and the above-mentioned operation 1 is an operation of clicking an emotion adjustment option such as a "relax" option, and the above-mentioned emotional cartoon character in a relaxed state (such as a state of relaxing on a bed, etc.) can be displayed in the above-mentioned video / animation.

[0350] In some examples, the video / animation described above can display an interface for guiding the user 1 to perform a mindfulness exercise.

[0351] The audio described above can be audio for adjusting the emotion of the user 1. Different emotional states can output different types of audio. For example, when the user 1 is in a positive emotional state, a hit song can be played; for another example, when the user 1 is in a neutral emotional state, R&B can be played; for yet another example, when the user 1 is in a negative or high-stress emotional state, light music can be played; and so on.

[0352] The running of the artificial intelligence chat described above can mean that the electronic device 100 incarnates as an artificial intelligence chat robot to guide the user 1 to have a conversation and pour out their feelings, so as to adjust the emotion of the user 1.

[0353] The running of the small game described above can mean that the electronic device 100 runs a game (such as the aforementioned knocking of wooden fish, the turning of a Buddhist rosary, etc.) for adjusting the emotion of the user 1, so as to divert the attention of the user 1 through the small game, thereby adjusting the emotion of the user 1.

[0354] Optionally, in a possible implementation, after the electronic device 100 performs the corresponding function based on the message 2, i.e., after S1604, the method can further include: the electronic device 100 sends a message 3 to the electronic device 200, the message 3 being used to instruct the electronic device 200 to display a virtual image in a second emotion; the electronic device 200 displays a third interface based on the message 3, the third interface including the virtual image in the second emotion; wherein the second emotion is the emotion of the user 1 detected by the electronic device 100 after performing S1604.

[0355] In this way, after detecting that the emotion of the user 1 changes from the first emotion to the second emotion, the electronic device 100 can notify the electronic device 200, so that the electronic device 200 can update the displayed virtual image.

[0356] In a possible implementation, the method further includes: the electronic device 100 acquires third data, the third data including one or more emotions of the user 1 generated in a second time period and factors associated with the one or more emotions, the factors including one or more of an activity track, schedule information, body movement, expression, and speech frequency of the user 1; and the electronic device 100 determines a personality type of the user 1 based on the third data. In this way, the personality type of the user can be inferred by analyzing the emotion of the user and the factors associated with the emotion of the user.

[0357] In a possible implementation, the method further includes: after detecting that the emotion of the user is a third emotion, the electronic device 100 can display a virtual image in the third emotion. The third emotion can be the first emotion or the second emotion described above. The electronic device 100 detects a second operation of the user 1 on the virtual image; and in response to the second operation, the electronic device 100 can perform a second function according to a first emotion regulation strategy. The first emotion regulation strategy is determined from a plurality of emotion regulation strategies based on a first condition, and the first condition includes one or more of the following: the first emotion regulation strategy is effective for the user, the personality type of the user is a first personality type, and the user is in a first scene. The first emotion regulation strategy includes one or more of the following: outputting a comfort interface, outputting a comfort audio, running an artificial intelligence chat, and running a game. The effectiveness includes that, after performing the second function, the electronic device 100 detects that the emotion of the user 1 changes to a fourth emotion, and the fourth emotion is a positive emotion relative to the third emotion. That is, after detecting the emotion of the user, the electronic device 100 can display a corresponding virtual image based on the detected emotion, and in response to an operation of the user on the virtual image, the electronic device 100 can comfort or respond to the emotion of the user 1.

[0358] In a possible implementation, the method further includes: the electronic device 100 predicts that a first event will occur at a first time, where the first event is a factor that triggers the user 1 to generate a fifth emotion; and the electronic device 100 outputs first prompt information, where the first prompt information is used to prompt the user 1 to regulate the emotion. In this way, the emotion of the user can be predicted in advance before an event occurs, and the user can be prompted in advance to adjust the emotion in time.

[0359] FIGS. 17A-17H exemplarily show another emotion display and interaction method provided by the embodiments of the present application. As shown in the figures, the method can include:

[0360] S1701, the electronic device 100 detects the emotion of the user.

[0361] Referring to the description in other embodiments of the present application, for example, the embodiments shown in FIGS. 2A, 2B, and 2C, the electronic device 100 can detect the emotion of the user through physiological data and the like of the user. The electronic device 100 can periodically or conditionally trigger to detect and update the emotion of the user, and can record the emotion of the user at a certain time point or time period, and can also record the change of the emotion of the user. Based on the detection of the electronic device 100, the emotion of the user is detected and recorded in real time or periodically.

[0362] S1702, when the trigger condition 1 is met, the electronic device 100 displays an interface A, where the interface A includes a first virtual image in state 1. The first virtual image in state 1 is used to indicate the user's mood at the current time of the electronic device 100.

[0363] The trigger condition 1 can be the trigger condition described in S1602 of the embodiment shown in the foregoing FIG. 16. For example, the trigger condition 1 can include: the electronic device is triggered to light up (such as lifting the wrist to light up, or pressing the crown to trigger the light up, or receiving a message notification to light up, etc.), or the electronic device detects that the user's mood has changed, for example, from positive mood (pleasure) to negative mood (unpleasant) and the like.

[0364] When the trigger condition 1 is met, the electronic device 100 displays a virtual image representing the user's current mood, and the state of the virtual image can represent the user's current mood. The user's current mood refers to the user's mood at time 1 when the trigger condition 1 is met. It can be the user's mood detected in real time at that time, or the most recent mood detection result. When the user's current mood is pleasure, the virtual image can display actions, expressions or animations of great pleasure, such as the animation shown in FIG. 17B, where the panda rolls happily; or as shown in FIG. 17C, where the panda listens to music happily. When the user's current mood is calm, the virtual image can display actions, expressions or animations of calm, such as shown in FIG. 17D or FIG. 17E, where the panda calmly lies on a branch or eats bamboo. When the user's current mood is unpleasant, the virtual image can display actions, expressions or animations of unpleasant, as shown in FIG. 17F.

[0365] Further, when displaying the interface A, the electronic device 100 can determine the color of the interface A based on the user's current mood in addition to displaying the virtual image corresponding to the state. Different colors can represent different moods, such as: yellow or orange for pleasure, green for calm, and blue or purple for unpleasant. For example, the interfaces shown in FIG. 17B and FIG. 17C can display a yellow or orange background color; the interfaces shown in FIG. 17D and FIG. 17E can display a green background color; and the interface shown in FIG. 17F can display a blue or purple background color.

[0366] S1703, when the trigger condition 2 is met, the electronic device 100 displays an interface B, where the interface B includes a first virtual image in state 2. The first virtual image in state 2 is used to indicate the user's mood at the current time of the electronic device 100.

[0367] The trigger condition 2 can also be the trigger condition described in S1602 of the embodiment shown in FIG. 16. The trigger condition 2 and the trigger condition 1 occur at different times, but the two trigger conditions can be the same or different. It should be noted that there can be other steps between S1702 and S1703. For example, the user can normally use the electronic device 100 between the two steps, such as turning off the screen, turning on the screen, checking messages, payment, and the like.

[0368] When the trigger condition 2 is met, the virtual avatar in state 2 displayed by the electronic device 100 represents the user's emotion at the time 2 when the trigger condition 2 is met. For example, it can be the user's emotion detected in real time at the time 2, or it can be the emotion detection result closest to the time 2. Based on the user's emotion, the virtual avatar in state 2 displayed by the electronic device 100 can also be seen in FIGS. 17B-17F.

[0369] Further, similar to the display interface A, when the electronic device 100 displays the interface B, the color of the interface B can also be determined based on the current emotion of the user.

[0370] It should be noted that when the user's emotion corresponding to the time (time 2) when the trigger condition 2 is met is different from the user's emotion corresponding to the time (time 1) when the trigger condition 1 is met, the state of the virtual avatar displayed by the electronic device 100 is different, that is, state 2 is different from state 1; further, the background color of the interface B is also different from the background color of the interface A. For example: at time 1, the user's emotion is happy, and the electronic device 100 can display the interface as shown in FIGS. 17B and 17C; at time 2, the user's emotion is not happy, and the electronic device 100 can display the interface as shown in FIG. 17F.

[0371] Further, the method can further include:

[0372] S1704, after the electronic device 100 displays the interface A or the interface B, the electronic device 100 can display the corresponding interface according to the current emotion of the user to adjust the emotion of the user.

[0373] For example: when the user's emotion is not happy, the electronic device 100 can display (1) of FIG. 17G; then according to the current emotion of the user, the electronic device can continue to display the animation as shown in (2)-(8) of FIG. 17G, wherein the panda rolls over and faces the user with a cute expression, which can have the effect of comforting the user. Or, when the user's emotion is not happy, the electronic device 100 can display (1) of FIG. 17H; then according to the current emotion of the user, the electronic device can continue to display the animation as shown in (2)-(6) of FIG. 17H, wherein the panda mom comes to comfort the panda baby, which can have the effect of adjusting the user's emotion.

[0374] Or, as described in the foregoing embodiments, the electronic device 100 can send an instruction to the electronic device 200 so that the electronic device 200 displays a virtual image representing the emotion of the user 1; after the electronic device 200 receives the operation of the virtual image by the user 2, the electronic device 200 can return an instruction to the electronic device 100, and the electronic device 100 displays a corresponding interface based on the returned instruction, which can represent the pacification or comfort of the user 2 to the emotion of the user 1. For example, when the emotion of the user is not happy, the electronic device 100 sends an instruction to the electronic device 200 so that the electronic device 200 displays an interface as shown in (1) of FIG. 17G or (1) of FIG. 17H; then the electronic device 200 returns an instruction to the electronic device 100 based on the operation of the user 2, and the electronic device 100 displays the interface as shown in (2)-(8) of FIG. 17G or (2)-(6) of FIG. 17H based on the returned instruction to represent the pacification of the user 2 to the user 1.

[0375] The structure of the electronic device 100 provided in an embodiment of the present application is described below.

[0376] FIG. 18 exemplarily shows the structure of the electronic device 100 provided in an embodiment of the present application.

[0377] As shown in FIG. 18, the electronic device 100 can include a processor 110, a memory 120, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna, a wireless communication module 160, an audio module 170, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and the like. The sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, a PPG sensor 180C, an acceleration sensor 180D, a distance sensor 180E, a proximity light sensor 180F, a touch sensor 180G, and the like.

[0378] It can be understood that the structure exemplified in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than those shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0379] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices or integrated in one or more processors.

[0380] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0381] The processor 110 can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can save instructions or data that the processor 110 has just used or repeatedly uses. If the processor 110 needs to use the instructions or data again, it can directly call from the memory. This avoids repeated access and reduces the waiting time of the processor 110, thereby improving the efficiency of the system.

[0382] In some embodiments, the processor 110 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, and / or a universal serial bus (USB) interface, etc.

[0383] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 can contain multiple sets of I2C buses. The processor 110 can be coupled to the touch sensor 180G, the charger, the flash, the camera 193, etc. through different I2C bus interfaces respectively. For example, the processor 110 can be coupled to the touch sensor 180G through an I2C interface, so that the processor 110 and the touch sensor 180G communicate through the I2C bus interface to realize the touch function of the electronic device 100.

[0384] The UART interface is a general serial data bus, which is used for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is usually used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to realize the Bluetooth function. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface to realize the function of playing music through the Bluetooth earphone.

[0385] The USB interface 130 is an interface that meets the USB standard specification, which can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect the charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and the peripheral device. It can also be used to connect the earphone to play audio through the earphone. The interface can also be used to connect other terminal devices, such as AR devices, etc.

[0386] It can be understood that the interface connection relationship between the modules shown in the embodiments of the present application is only illustrative and does not constitute a structural limitation of the electronic device 100. In some other embodiments of the present application, the electronic device 100 can also use different interface connection methods or combinations of multiple interface connection methods in the above embodiments.

[0387] The wireless communication module 160 can provide a solution for wireless communication, including wireless local area networks (WLAN) (e.g., wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc., which are applied on the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via an antenna, frequency-modulates and filters the electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 can also receive signals to be transmitted from the processor 110, frequency-modulate them, amplify them, and radiate them as electromagnetic waves via an antenna.

[0388] The electronic device 100 implements a display function through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, which is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs that execute program instructions to generate or change display information.

[0389] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flex light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diode (QLED), etc. In some embodiments, the electronic device 100 can include 1 or N display screens 194, and N is a positive integer greater than 1.

[0390] The electronic device 100 can implement a photographing function through an ISP, the camera 193, a video codec, a GPU, the display 194, and an application processor.

[0391] The camera 193 is configured to capture still images or videos. An object projects an optical image through a lens to a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then transmitted to an ISP to be converted into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into an image signal in a standard format, such as RGB, YUV, or the like. In some embodiments, the electronic device 100 can include one or N cameras 193, where N is a positive integer greater than one.

[0392] The electronic device 100 can implement an audio function through the audio module 170 and an application processor. For example, music playing, voice recording, and the like.

[0393] The audio module 170 is configured to convert digital audio information into an analog audio signal for output, and to convert an analog audio input into a digital audio signal. The audio module 170 can also be configured to encode and decode audio signals. In some embodiments, the audio module 170 can be disposed in the processor 110, or some of the functional modules of the audio module 170 can be disposed in the processor 110.

[0394] The pressure sensor 180A is configured to sense a pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed in the display 194. There are many types of pressure sensors 180A, such as a resistive pressure sensor, an inductive pressure sensor, a capacitive pressure sensor, and the like. A capacitive pressure sensor can include at least two parallel plates made of conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure based on the change in capacitance. When a touch operation is applied to the display 194, the electronic device 100 detects the intensity of the touch operation based on the pressure sensor 180A. The electronic device 100 can also calculate the position of the touch based on the detection signal of the pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation instructions. For example, when a touch operation with a touch operation intensity less than a first pressure threshold is applied to a short message application icon, an instruction to view a short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold is applied to the short message application icon, an instruction to create a new short message is executed.

[0395] The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of shaking of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and lets the lens offset the shaking of the electronic device 100 by reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and motion sensing game scenarios.

[0396] The PPG sensor 180C (also referred to as a PPG module) can be used to collect physiological data of a user (e.g., heart rate, blood oxygen, respiration rate, blood pressure, blood sugar, muscle oxygen, vascular resistance, brain oxygen, etc.). In embodiments of the present application, the electronic device 100 can detect the mood of the user based on the collected physiological data of the user.

[0397] The acceleration sensor 180D can detect the magnitude of acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device 100 and applied to landscape / portrait switching, pedometer, etc.

[0398] The distance sensor 180E is used to measure distance. The electronic device 100 can measure distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180E to measure distance to achieve fast focusing.

[0399] The proximity light sensor 180F can include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The light-emitting diode can be an infrared light-emitting diode. The electronic device 100 emits infrared light outwardly through the light-emitting diode. The electronic device 100 detects infrared reflected light from nearby objects using the photodiode. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100.

[0400] The touch sensor 180G, also referred to as a "touch panel". The touch sensor 180G can be disposed on the display screen 194, and the touch sensor 180G and the display screen 194 form a touch screen, also referred to as a "touch panel". The touch sensor 180G is used to detect a touch operation acting on or near the touch sensor 180G. The touch sensor 180G can transmit the detected touch operation to the application processor to determine the touch event type. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180G can also be disposed on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0401] The keys 190 include a power-on key, a volume key, and the like. The keys 190 can be mechanical keys. They can also be touch keys. The electronic device 100 can receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0402] The motor 191 can generate a vibration prompt. The motor 191 can be used for touch vibration feedback. For example, a touch operation acting on different applications (such as taking a photo, playing audio, and the like) can correspond to different vibration feedback effects. A touch operation acting on different regions of the display screen 194 can also correspond to different vibration feedback effects. Different application scenarios (such as time reminders, receiving messages, alarms, games, and the like) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0403] The indicator 192 can be an indicator light, which can be used to indicate a charging state, a power change, and can also be used to indicate a message, a notification, and the like.

[0404] It should be understood that the electronic device 100 shown in FIG. 18 is only an example, and the electronic device 100 can have more or fewer components than those shown in FIG. 18, can combine two or more components, or can have a different component configuration. The various components shown in FIG. 18 can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0405] The structure of the electronic device 200 can be the same as or similar to that of the electronic device 100, and the related content about the structure of the electronic device 200 can refer to the related textual description of the structure of the electronic device 100 shown in FIG. 18, which will not be described here.

[0406] The software structure of the electronic device 100 provided in the embodiments of the present application is described below.

[0407] FIG. 19 exemplarily shows a software structure of an electronic device 100 provided in the embodiments of the present application.

[0408] As shown in FIG. 19, the software system of the electronic device 100 can employ a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. Embodiments of the present application take an Android system with a layered architecture as an example to exemplarily illustrate the software structure of the electronic device 100.

[0409] The layered architecture divides software into several layers, each of which has a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, an application layer, an application framework layer, a system library, and a kernel layer.

[0410] The application layer can include a series of application packages.

[0411] As shown in FIG. 19, the application packages can include clock, mood, calendar, call, map, settings, WLAN, Bluetooth, music, video, short message, and the like.

[0412] Among them, the mood application is an application installed on the electronic device 100 for mood detection and mood regulation.

[0413] In embodiments of the present application, the mood application can be used to monitor the mood of the user based on the results detected by various sensors, and can perform multi-classification on the mood (such as positive mood, neutral mood, negative mood, etc.), and generate a mood detection result.

[0414] It can be understood that the mood detection result generated by the mood application can be used as a parameter for generating a virtual image. In a possible implementation, the virtual image can be used to indicate the mood detection result. It can be understood that the description of the virtual image here can refer to the description in the foregoing embodiments.

[0415] In embodiments of the present application, the mood application can also display a virtual image representing the mood of the user. That is, the virtual image can be displayed in the electronic device 100 in response to the user's operation of opening the mood application. In a possible implementation, the virtual image can be generated by the mood application and displayed. In another possible embodiment, for example, the virtual image representing the mood of the user can be generated in the following virtual image generation module, and the mood application can obtain information of the virtual image and display the virtual image based on the information.

[0416] In the embodiments of the present application, the mood application program can also be used to provide mood regulation strategies, so that the user can regulate his mood through the mood regulation strategies. The mood regulation strategies can include, but are not limited to, video / animation, audio, artificial intelligence chat, mindfulness practice, and small games. Specifically, the mood regulation strategies can refer to the description in the foregoing embodiments, which will not be repeated here.

[0417] It should be noted that the name of the mood application program is only a word used in the embodiments of the present application, and the meaning represented thereby has been described in the embodiments of the present application. The name does not constitute any limitation on the embodiments of the present application.

[0418] The application program framework layer provides an application programming interface (API) and a programming framework for the application programs of the application program layer. The application program framework layer includes some pre-defined functions.

[0419] As shown in FIG. 19, the application program framework layer can include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0420] The window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, and intercept the screen.

[0421] The content provider is used to store and obtain data, and make the data accessible to the application programs. The data can include videos, images, audios, dialed and received calls, browsing history and bookmarks, phone books, and the like.

[0422] The view system includes visual controls, such as controls for displaying text and controls for displaying pictures. The view system can be used to build application programs. A display interface can be composed of one or more views. For example, a display interface including a short message notification icon can include a view for displaying text and a view for displaying pictures.

[0423] The phone manager is used to provide the communication function of the electronic device 100. For example, the management of the call state (including call connection, call hang-up, and the like).

[0424] The resource manager provides various resources for the application programs, such as localized strings, icons, pictures, layout files, video files, and the like.

[0425] The notification manager enables applications to display notification information in the status bar, which can be used to convey alert-type messages that can automatically disappear after a brief stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the form of a figure or a scroll bar text in the top status bar of the system, such as a notification of an application running in the background, and can also be a notification that appears in the form of a dialog window on the screen. For example, the status bar prompts text information, issues a prompt sound, the electronic device vibrates, the indicator light flashes, etc.

[0426] The core library includes two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android.

[0427] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java files of the application layer and the application framework layer into binary files. The virtual machine is used to perform functions such as management of object life cycle, stack management, thread management, security and exception management, and garbage collection.

[0428] The system library can include multiple functional modules. For example: surface manager, media library, three-dimensional graphics processing library (for example: OpenGL ES), 2D graphics engine (for example: SGL), virtual image generation module, etc.

[0429] The surface manager is used to manage the display subsystem and provides 2D and 3D layer fusion for multiple applications.

[0430] The media library supports multiple commonly used audio, video format playback and recording, and static image files, etc. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0431] The three-dimensional graphics processing library is used to realize three-dimensional graphics drawing, image rendering, synthesis, and layer processing, etc.

[0432] The 2D graphics engine is a drawing engine for 2D drawing.

[0433] The virtual image generation module can generate a virtual image. Specifically, the virtual image generation module can be configured to acquire one or more parameters and generate a virtual image based on the acquired one or more parameters. For example, the virtual image generation module can be configured to acquire the emotion detection result from the emotion application, and generate a corresponding virtual image based on the emotion detection result and the prompt information described above, a state of the virtual image can be used to indicate the notification message and the emotion of the user. The virtual image generation module can also be configured to receive the notification message sent by the notification manager, and generate a corresponding virtual image based on the notification message, a state of the virtual image can be used to indicate the notification message.

[0434] In some examples, the virtual image generation module can also generate a virtual image based only on the emotion detection result acquired in the emotion application.

[0435] In other examples, the virtual image generation module can also generate a virtual image based on the emotion detection result acquired in the emotion application and personalized information of the user (such as environmental information of the user, sports health data, associated event information, personal information of the user, etc.).

[0436] The virtual image generation module can also be configured to generate a virtual image when a preset condition is met. For example, the virtual image generation module can generate a virtual image when a trigger condition (or trigger event) described in the foregoing embodiments is met; the virtual image generation module can also be configured to update a state of the virtual image after the user interacts with the virtual image; the virtual image generation module can also be configured to acquire an emotion regulation strategy from the emotion application, and determine an emotion regulation strategy matched with a current scene based on the emotion regulation strategy and the current scene, to prompt the user to use the emotion regulation strategy to regulate his / her emotion; and the like.

[0437] After the virtual image generation module generates a corresponding virtual image, a display driver in the kernel layer can be started, so that the virtual image and / or content related to the emotion regulation strategy can be displayed on a display screen. For details, refer to the foregoing description, which will not be repeated here.

[0438] The kernel layer is a layer between hardware and software. The kernel layer at least includes a display driver, a camera driver, a Bluetooth driver, and a sensor driver.

[0439] Embodiments of the present application provide a chip system, comprising: a processor coupled with a memory, the memory being configured to store programs or instructions, when the programs or instructions are executed by the processor, the chip system implements the method in any of the method embodiments described above.

[0440] Optionally, the processor in the chip system can be one or more. The processor can be implemented by hardware or software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented by software, the processor can be a general-purpose processor, which is implemented by reading software codes stored in a memory.

[0441] Optionally, the memory in the chip system can also be one or more. The memory can be integrated with the processor or arranged separately from the processor, and the embodiments of the present application do not limit the same. Illustratively, the memory can be a non-transient processor, for example, a read-only memory (ROM), which can be integrated on the same chip as the processor or arranged on different chips respectively, and the embodiments of the present application do not limit the type of the memory and the arrangement of the memory and the processor.

[0442] Illustratively, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), a micro controller unit (MCU), a programmable logic device (PLD) or other integrated chip.

[0443] It should be understood that each step in the above method embodiments can be completed by the integrated logic circuit of hardware in the processor or the instructions in the form of software. The method steps disclosed in conjunction with the embodiments of the present application can be directly embodied as hardware processor execution completion or executed by a combination of hardware and software modules in the processor.

[0444] In the above embodiments, all or part of the processes can be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the processes can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes described in the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes one or more available media sets. The available media can be magnetic media (such as floppy disks, hard disks, magnetic tapes), optical media (such as DVDs), or semiconductor media (such as solid state disks (SSD)), etc.

[0445] Those of ordinary skill in the art can understand that all or part of the processes in the above embodiments can be instructed by a computer program to complete the relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned method embodiments. The aforementioned storage medium includes ROM or random access memory (RAM), magnetic disk or optical disk, and various media that can store program codes.

[0446] The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

An interaction method applied to a communication system comprising a first electronic device and a second electronic device, characterized in that, The method comprises: The first electronic device detects the emotion of the first user; The first electronic device sends a first message to the second electronic device, the first message being used to instruct the second electronic device to display a first virtual image in a first emotion, the first emotion being the emotion of the first user detected by the first electronic device; The second electronic device displays a first interface, and the first interface includes the first virtual image in the first emotion; The second electronic device detects a first operation of a second user on the first virtual image; In response to the first operation, the second electronic device sends a second message to the first electronic device; The first electronic device receives the second message and performs a first function based on the second message; The first function includes one or more of the following: outputting a second interface, outputting a first audio, running an artificial intelligence chat, and running a first game. The method of claim 1, wherein The first operation is an operation on the first virtual image; Or, The first operation is an operation on a function control included in the first interface. The method according to claim 1 or 2, characterized in that The first emotion is a negative emotion, the first operation is an operation for calming the first user, and the second interface is an interface for calming the first virtual image. The method according to any one of claims 1 to 3, characterized in that The second interface includes the first virtual image in a first state corresponding to the first operation. The method according to any one of claims 1 to 4, characterized in that After the first electronic device performs the first function based on the second message, the method further comprises: The first electronic device sends a third message to the second electronic device; The second electronic device displays a third interface based on the third message, and the third interface includes the first virtual image in a second emotion, the second emotion being the emotion of the first user detected by the first electronic device after performing the first function. The method of any one of claims 1-5, wherein The first virtual image is generated based on first data, and the first data is physiological data of the first user used to determine the emotion; Or, The first virtual image is generated based on the first data and second data, and the second data is personalized information of the first user, the personalized information of the first user including one or more of the following: environmental information of the first user, exercise health data, event information associated with generating the first emotion, and personal information of the first user. The method according to any one of claims 1 to 6, characterized in that The first interface is displayed full screen, or the first interface is displayed in the form of a card. The method according to any one of claims 1 to 7, characterized in that Before the second electronic device displays the first interface, the method further comprises: The first electronic device detects an operation of the first user granting the second electronic device permission to display the first virtual image. The method of any one of claims 1-8, wherein The first electronic device sending the first message to the second electronic device comprises: the first electronic device sending the first message to the second electronic device in response to a first trigger event; wherein the first trigger event comprises one or more of the following: A specific user action; A specific scene; A specific time or time period; A specific wake-up word; A change in the emotion detection result of the first user meets a first preset condition; A change in the environmental information in which the first user is located meets a second preset condition; A change in the motion health data of the first user meets a third preset condition. An interaction method applied to a second electronic device, characterized in that, The method comprises: The second electronic device receiving the first message sent by the first electronic device; In response to the first message, the second electronic device displays a first interface, and the first interface includes a first virtual image in a first emotion, which is the emotion of the first user detected by the first electronic device; The second electronic device detects a first operation of the second user on the first virtual image; In response to the first operation, the second electronic device sends a second message to the first electronic device, and the second message is used to instruct the first electronic device to perform a first function; Wherein, the first function comprises one or more of the following: outputting a second interface, outputting a first audio, running an artificial intelligence chat, and running a first game. According to the method of claim 10, wherein The first operation is an operation on the first virtual image; Or, The first operation is an option of a function control included in the first interface. The method according to claim 10 or 11, characterized in that The first emotion is a negative emotion, the first operation is an operation for soothing the first user, and the second interface is an interface for soothing the first virtual image. The method according to any one of claims 10-12, characterized in that The second interface includes the first virtual image in a first state, and the first state corresponds to the first operation. The method according to any one of claims 10-13, characterized in that After the second electronic device sends the second message to the first electronic device, the method further comprises: The second electronic device receiving a third message sent by the first electronic device; The second electronic device displays a third interface based on the third message, and the third interface includes the first virtual image in a second emotion, which is the emotion of the first user detected by the first electronic device after performing the first function. According to the method of any one of claims 10-14, wherein The first virtual image is generated based on first data, and the first data is physiological data of the first user for determining emotion; Or, The first virtual image is generated based on the first data and second data, and the second data is personalized information of the first user, and the personalized information of the first user comprises one or more of the following: environmental information in which the first user is located, motion health data, event information associated with generating the first emotion, and personal information of the first user. The method according to any one of claims 10-15, characterized in that The first interface is displayed full screen, or the first interface is displayed in the form of a card. The method according to any one of claims 10-16, characterized in that The second electronic device is granted a permission to display the first virtual image. The method of claim 17, wherein The permission to display the first virtual image specifically includes: The permission to display the first virtual image in the fourth interface, the fourth interface including one or more of a minus one screen interface, a lock screen interface, a desktop, an off-screen display interface, and an application interface. And / or, The permission to display the first virtual image in the first time period. An interaction method applied to a first electronic device, characterized in that, The method includes: The first electronic device detects the emotion of the first user. The first electronic device sends a first message to a second electronic device, the first message indicating the second electronic device to display a first virtual image in a first emotion, the first emotion being the emotion of the first user detected by the first electronic device. The first electronic device receives a second message sent by the second electronic device. The first electronic device performs a first function based on the second message. The execution of the first function includes one or more of the following: outputting a second interface, outputting a first audio, running an artificial intelligence chat, and running a first game. The method of claim 19, wherein The second interface includes the first virtual image in a first state, the first state being determined based on the second message. The method according to claim 19 or 20, characterized in that Before the first electronic device sends a second message to the second electronic device, the method further includes: The first electronic device obtains first data and generates the first virtual image based on the first data, the first data being physiological data of the first user for determining the emotion. Or, The first electronic device obtains the first data and second data, and generates the first virtual image based on the first data and the second data, the second data being personalized information of the first user, the personalized information of the first user including one or more of the following: environmental information of the first user, exercise health data, event information associated with generating the first emotion, and personal information of the first user. The method according to any one of claims 20-21, characterized in that Before the first electronic device sends a first message to the second electronic device, the method further includes: The first electronic device detects an operation of the first user granting the second electronic device a permission to display the first virtual image. The method of claim 22, wherein The first electronic device detects an operation of the first user granting the second electronic device a permission to display the first virtual image, specifically including: The first electronic device detects an operation of the first user granting the second electronic device a permission to display the first virtual image in the first interface, the first interface including one or more of a minus one screen interface, a lock screen interface, a desktop, an off-screen display interface, and an application interface. And / or, The first electronic device detects an operation of the first user granting the second electronic device a permission to display the first virtual image in a first time period. The method according to any one of claims 21-23, characterized in that The method further includes: The first electronic device obtains third data, the third data including one or more emotions of the first user generated in a second time period and factors associated with the one or more emotions, the factors including one or more of an activity track, schedule information, body movement, expression, and speech frequency of the first user; The first electronic device determines a personality type of the first user based on the third data. The method according to any one of claims 19-24, characterized in that After the first electronic device performs the first function based on the second message, the method further includes: The first electronic device detects that the emotion of the first user changes to a second emotion; The first electronic device sends a third message to the second electronic device; the third message is used to instruct the second electronic device to display the first virtual image in the second emotion. The method according to any one of claims 19-25, characterized in that The method further includes: The first electronic device displays the first virtual image in a third emotion, the third emotion being the emotion of the first user detected by the first electronic device; The first electronic device detects a second operation of the first user on the first virtual image; In response to the second operation, the first electronic device performs a second function, the performance of the second function being based on a first emotion regulation strategy, the first emotion regulation strategy being determined from a plurality of emotion regulation strategies based on a first condition, the first condition including one or more of: the first emotion regulation strategy being effective for the first user, the personality type of the first user being a first personality type, the first user being in a first scenario; The effective includes: after performing the second function, the first electronic device detects that the emotion of the first user changes to a fourth emotion, the fourth emotion being a positive emotion relative to the third emotion. The method of claim 26, wherein The first emotion regulation strategy includes one or more of: outputting a soothing interface, outputting soothing audio, running an artificial intelligence chat, and running a game. The method according to any one of claims 19-27, characterized in that The method further includes: The first electronic device predicts that a first event will occur at a first time, wherein the first event is a factor that triggers the first user to generate a fourth emotion; The first electronic device outputs first prompt information, the first prompt information being used to prompt the first user to regulate emotions. The method of any one of claims 19-28, wherein The first electronic device sending the first message to the second electronic device includes: in response to a first trigger event, the first electronic device sending the first message to the second electronic device; wherein the first trigger event includes one or more of: A specific user action; A specific scenario; A specific time or time period; A specific wake-up word; A change in the emotion detection result of the first user meets a first preset condition; A change in the environmental information in which the first user is located meets a second preset condition; A change in the motion health data of the first user meets a third preset condition. A method of displaying an emotion, characterized by The method further includes: The first electronic device detects a situation of the first user; When the first trigger condition is met, the first electronic device displays a first interface, the first interface including a first virtual image in a first state; the first virtual image in the first state is used to indicate a first emotion of the first user when the first trigger condition is met; When the second trigger condition is met, the first electronic device displays a second interface, the second interface including a first virtual image in a second state; The first virtual image in the second state is used to indicate a second emotion of the first user when the second trigger condition is met. The method of claim 30, wherein: The first interface presents a first color corresponding to the first emotion; The second interface presents a second color corresponding to the second emotion. The method of claim 30 or 31, wherein: When the first emotion is different from the second emotion, the first state is different from the second state; or When the first emotion is different from the second emotion, the first color is different from the second color. An electronic device, characterized by comprising: The electronic device includes one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, the one or more memories are configured to store computer program codes, the computer program codes include computer instructions, when the one or more processors execute the computer instructions, the electronic device is caused to perform the method of any one of claims 10-32. A computer storage medium, characterized by The computer storage medium stores a computer program, the computer program includes program instructions, when the program instructions run on an electronic device, the electronic device is caused to perform the method of any one of claims 10-32. A computer program product, when the computer program product runs on a computer, causes the computer to perform the method of any one of claims 10-32.

Images