Emotion detection method and electronic device

Abstract

The present application provides an emotion detection method and an electronic device. In the method, the electronic device acquires first physiological data of a user at a first moment, and generates first physiological feature information according to the first physiological data; the electronic device generates a first emotion result corresponding to the first moment on the basis of a target emotion detection model according to the first physiological feature information. A training sample set used for obtaining the target emotion detection model by training is generated according to the physiological data of the user and the emotion results corresponding to the physiological data. The emotion results corresponding to the physiological data comprise the emotion results after calibration by the user and / or the emotion results after calibration by a historical emotion detection model. The historical emotion detection model includes at least one emotion detection model used prior to the target emotion detection model in the emotion detection process. By means of the solution, the target detection model can better fit the physiological and psychological conditions of the user, thereby achieving personalized emotion detection and improving the detection accuracy of the model.

Description

An emotion detection method and electronic device

[0001] Cross-reference of related applications

[0002] This application claims priority to Chinese Patent Application No. 202410700383.7, filed on May 30, 2024, entitled "A Method and Electronic Device for Emotion Detection", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of terminal technology, and in particular to an emotion detection method and electronic device. Background Technology

[0004] With the development of terminal technology, in addition to providing users with communication and entertainment services, terminal devices can also monitor users' health. For example, wearable devices can monitor users' heart rate and exercise. Since users' emotions can affect their work and life, it is crucial to help users discover potential emotional problems as early as possible. Summary of the Invention

[0005] This application provides an emotion detection method and an electronic device, which provide a personalized method for detecting user emotions, thereby improving the accuracy of emotion detection.

[0006] Firstly, this application provides an emotion detection method, which can be executed by an electronic device. In this method, the electronic device acquires first physiological data of a user at a first moment, and generates first physiological feature information based on the first physiological data; the electronic device generates a first emotion result corresponding to the first moment based on the first physiological feature information and a target emotion detection model; wherein the target emotion detection model is trained based on a training sample set corresponding to the user's user data, the training sample set corresponding to the user data is generated based on the user's physiological data and the emotion result corresponding to the physiological data, the emotion result corresponding to the physiological data includes the user-calibrated emotion result, and / or, the emotion result calibrated by a historical emotion detection model, the historical emotion detection model including at least one emotion detection model used before the target emotion detection model during the emotion detection process.

[0007] In the above methods, the target detection model used by electronic devices to detect users' emotions is trained based on the user's calibrated emotion results and / or the calibrated emotion results of historical emotion detection models. This makes the target detection model more closely match the user's physiological and psychological state, thereby achieving personalized emotion detection and improving the model's detection accuracy.

[0008] In one possible design, after generating the first emotion result corresponding to the first moment based on the first physiological feature information and the target emotion detection model, the method further includes: displaying the first emotion result corresponding to the first moment; displaying the first emotion result corresponding to the first moment; displaying an emotion result editing interface in response to a first user operation; and changing the emotion result corresponding to the first moment from the first emotion result to a second emotion result in response to a second user operation on the emotion result editing interface.

[0009] With this design, after detecting the user's emotional state, the electronic device can display the emotional state result, and the user can modify the detected emotional state result to calibrate it.

[0010] In one possible design, the method further includes: displaying a first reminder message, the first reminder message being used to ask the user whether to update the training of the target emotion detection model based on the user-calibrated emotion results; in response to a third operation by the user, generating a first training sample based on the first physiological data and the second emotion results, updating the training of the target emotion detection model based on a training sample set including the first training sample to obtain an updated target emotion detection model, and replacing the target emotion detection model with the updated target emotion detection model.

[0011] With this design, after the user calibrates the emotion results, the electronic device can ask the user whether to update and train the target emotion detection model based on the calibrated emotion results. After the user confirms, the electronic device can update and train the target emotion detection model based on the calibrated emotion results, thus providing a method for user-triggered model updates. This allows for timely updates to the model used in the emotion detection process, further improving the accuracy of the model in emotion detection.

[0012] In one possible design, the method further includes: determining that the target emotion detection model has been updated, displaying a second reminder message, the second reminder message being used to ask the user whether to calibrate uncalibrated emotion results among multiple emotion results corresponding to multiple moments stored in the electronic device based on the updated target emotion detection model; and calibrating the uncalibrated emotion results based on the updated target emotion detection model in response to a fourth operation triggered by the user.

[0013] With this design, once the electronic device obtains the updated target emotion detection model, it can also calibrate the uncalibrated emotion results among the multiple emotions stored in the electronic device based on the updated target emotion detection model, thereby providing users with more accurate emotion results.

[0014] In one possible design, the uncalibrated emotional outcome is at least one emotional outcome selected by the user.

[0015] This design allows users to select which emotional outcomes to calibrate based on the updated target emotion detection model, enabling electronic devices to calibrate the selected emotional outcomes to better suit user needs.

[0016] In one possible design, generating the first emotion result corresponding to the first moment based on the first physiological feature information and the target emotion detection model includes: generating a third emotion result based on the first physiological feature information and the target emotion detection model, and generating a fourth emotion result based on the first physiological feature information and the general emotion detection model; and determining the first emotion result based on the third emotion result and the fourth emotion result.

[0017] Through this design, electronic devices can perform emotion detection using a general emotion detection model and a target emotion detection model based on the user's primary physiological characteristics. The emotion results output by the two models are then fused to obtain the final emotion result. This approach considers both the relationship between general physiological characteristics and emotions, as well as the relationship between the user's individual physiological characteristics and emotions, resulting in more accurate emotion detection results.

[0018] In one possible design, the third emotional result includes a first probability value corresponding to each of the multiple emotions; the fourth emotional result includes a second probability value corresponding to each of the multiple emotions; determining the first emotional result based on the third emotional result and the fourth emotional result includes: calculating the average probability value corresponding to each of the multiple emotions based on the third emotional result and the fourth emotional result, and taking the emotion with the highest average probability value among the multiple emotions as the first emotional result.

[0019] In one possible design, the emotional results corresponding to the physiological data may also include emotional results generated by a general emotion detection model, and / or emotional results generated by the historical emotion detection model.

[0020] With this design, the training samples used by the electronic device when training the target emotion detection model can also include emotion results generated by a general emotion detection model, and / or emotion results generated by the historical emotion detection model, thereby enriching the training samples. When a large number of training samples are required, the training of the target emotion detection model can be completed as soon as possible, improving efficiency.

[0021] In one possible design, the first physiological data is the heart rate interval (RRI); generating the first physiological feature information based on the first physiological data includes: calculating the user's heart rate variability (HRV) feature based on the first physiological data; calculating the user's feature baseline based on the user's historical physiological data; and normalizing the HRV feature based on the feature baseline to obtain the first physiological feature information.

[0022] Through this design, when generating physiological feature information based on the user's physiological data, electronic devices can normalize the HRV features based on the feature baseline. The normalized physiological feature information can ensure that the numerical differences and fluctuations between different features are within a controllable range of the same order of magnitude, thereby improving the robustness and generalization ability of the emotion detection model.

[0023] In one possible design, the user's HRV characteristics include at least one of the following: mean RR interval, standard deviation of RR interval, standard deviation of adjacent RRIs, median RRI, proportion of RRI intervals over 50ms, and proportion of RRI intervals over 20ms.

[0024] Through this design, electronic devices can generate various types of physiological feature information based on the user's physiological data, thereby enriching the relationship between the physiological feature information that the model can learn during model training and the user's emotions, and further improving the accuracy of the model in emotion detection.

[0025] In one possible design, the target emotion detection model is obtained by training an initial model based on the training sample set, or the target emotion detection model is obtained by training and updating the historical emotion detection model based on the training sample set.

[0026] This design allows electronic devices to train an initial model to obtain a target emotion detection model, and also to update and train the target emotion detection model. This ensures that the target emotion detection model accurately reflects the user's physiological and psychological state during emotion detection. For example, the electronic device trains an initial model to obtain target emotion detection model 0. It can then update and train target emotion detection model 0 to obtain target emotion detection model 1, replacing target emotion detection model 0, and so on, thus achieving continuous updating of the target emotion detection model.

[0027] Secondly, this application provides an electronic device comprising multiple functional modules; the multiple functional modules interact to implement the methods performed by the electronic device in any of the above aspects and their respective embodiments. The multiple functional modules can be implemented based on software, hardware, or a combination of software and hardware, and the multiple functional modules can be arbitrarily combined or divided based on specific implementations.

[0028] Thirdly, this application provides an electronic device including at least one processor and at least one memory, wherein the at least one memory stores computer program instructions, and when the electronic device is running, the at least one processor executes any of the above aspects and the methods executed by the electronic device in its various embodiments.

[0029] Fourthly, this application also provides a computer program product containing instructions that, when the computer program product is run on a computer, cause the computer to perform the method executed by the server or electronic device in any of the above aspects and their respective embodiments.

[0030] Fifthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a computer, causes the computer to perform the method executed by the server or electronic device in any of the above aspects and embodiments.

[0031] Sixthly, this application also provides a chip for reading a computer program stored in a memory and executing the method performed by a server or electronic device in any of the above aspects and their embodiments.

[0032] Seventhly, this application also provides a chip system including a processor for supporting a computer device in implementing the methods performed by a server or electronic device in any of the above aspects and their embodiments. In one possible design, the chip system further includes a memory for storing programs and data necessary for the computer device. The chip system may be composed of chips or may include chips and other discrete devices. Attached Figure Description

[0033] Figure 1 is a schematic diagram of the scenarios in which the emotion detection method provided in the embodiments of this application is applicable;

[0034] Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0035] Figure 3 is a software structure block diagram of an electronic device provided in an embodiment of this application;

[0036] Figure 4 is a schematic diagram of an emotion result display interface of an electronic device provided in an embodiment of this application;

[0037] Figure 5 is a schematic diagram of an electronic device displaying emotion results according to an embodiment of this application;

[0038] Figure 6 is a flowchart of an emotion detection method provided in an embodiment of this application;

[0039] Figure 7 is a flowchart of an emotion detection method provided in an embodiment of this application;

[0040] Figure 8 is a schematic diagram of an interface for updating an emotion detection model provided in an embodiment of this application;

[0041] Figure 9 is a schematic diagram of an interface that asks the user whether to calibrate historical sentiment results according to an embodiment of this application;

[0042] Figure 10 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0043] Figure 11 is a flowchart of an emotion detection method provided in an embodiment of this application;

[0044] Figure 12 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0045] Figure 13 is a flowchart illustrating an emotion detection method provided in an embodiment of this application. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings. In the description of the embodiments of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature.

[0047] It should be understood that in the embodiments of this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0048] Emotion detection technology, also known as emotion recognition technology, detects a user's emotions by collecting physiological data through wearable devices. When the user wears the device continuously for a period of time while in a non-moving state, the wearable device or associated electronic devices input the physiological characteristic information corresponding to the collected data into a general emotion detection model, obtaining the emotional result output by the model. For example, emotional results may include pleasure, good mood, calmness, anxiety, and sadness. During the emotion detection process, the user only needs to wear the wearable device, and the user is unaware of the detection process.

[0049] However, when using the aforementioned emotion detection technology to detect user emotions, the emotion results obtained based on the general emotion detection model cannot adapt to the physiological and psychological differences between different individuals. As a result, the detected emotion results are more accurate for some users, but biased for others.

[0050] To address the aforementioned issues, this application provides an emotion detection method and an electronic device, offering a personalized emotion detection approach. Figure 1 illustrates a scenario applicable to the emotion detection method provided in this application. Referring to Figure 1, the emotion detection method provided in this application can be applied to wearable devices. In practice, the wearable device can collect the user's first physiological data at a first moment and generate first physiological feature information based on the user's first physiological data. The wearable device can then generate a first emotion result corresponding to the first moment based on the user's first physiological feature information and a target emotion detection model. The target emotion detection model is trained using a set of training samples corresponding to the user's data. This set of training samples is generated based on the user's physiological data and the corresponding emotion results. The emotion results corresponding to the physiological data include the user-calibrated emotion results and / or the emotion results calibrated using historical emotion detection models. The historical emotion detection models include at least one emotion detection model used before the target emotion detection model during the emotion detection process. This target detection model can better match the user's physiological and psychological state, achieving personalized emotion detection.

[0051] Optionally, the emotion detection method provided in this application embodiment can also be executed by a wearable device and an electronic device associated with the wearable device, such as a mobile phone or tablet computer. After collecting the user's physiological data, the wearable device can send the user's physiological data to the electronic device associated with the wearable device. The electronic device associated with the wearable device generates physiological feature information based on the user's physiological data, and performs emotion detection based on the target emotion detection model according to the physiological feature information. The electronic device then sends the detected emotion result back to the wearable device.

[0052] Optionally, the emotion detection method provided in this application embodiment can also be executed by a wearable device and a server. After collecting the user's physiological data, the wearable device can send the user's physiological data to the server. The server generates physiological feature information based on the user's physiological data and performs emotion detection based on the target emotion detection model according to the physiological feature information. The server then sends the detected emotion result to the wearable device. Alternatively, the emotion detection method provided in this application embodiment can also be executed by a wearable device, an electronic device associated with the wearable device, and a server. In this case, the electronic device associated with the wearable device and the server can collaboratively complete the emotion detection process.

[0053] For ease of explanation, the following embodiments use an electronic device executing the emotion detection method provided in this application as an example. The electronic device and embodiments for using such an electronic device are described below. The electronic device in this application embodiment can be a wearable device, tablet computer, mobile phone, in-vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc. This application embodiment does not impose any limitations on the specific type of electronic device.

[0054] In some embodiments of this application, the electronic device may also be a portable terminal device that includes other functions such as a personal digital assistant and / or a music player. Exemplary embodiments of the portable terminal device include, but are not limited to, devices equipped with... Or portable terminal devices with other operating systems.

[0055] Figure 2 is a schematic diagram of the structure of an electronic device 100 provided in an embodiment of this application. As shown in Figure 2, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0056] Processor 110 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. The controller may serve as the central nervous system and command center of the electronic device 100. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution. Processor 110 may also include memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that processor 110 has recently used or is repeatedly used. If processor 110 needs to reuse an instruction or data, it can directly retrieve it from the memory. This avoids repeated access, reduces the waiting time of processor 110, and thus improves system efficiency.

[0057] USB interface 130 is a USB standard compliant interface, specifically a Mini USB interface, Micro USB interface, USB Type-C interface, etc. USB interface 130 can be used to connect a charger to charge electronic device 100, and can also be used for data transfer between electronic device 100 and peripheral devices. Charging management module 140 receives charging input from the charger. Power management module 141 connects battery 142, charging management module 140, and processor 110. Power management module 141 receives input from battery 142 and / or charging management module 140, providing power to processor 110, internal memory 121, external memory, display 194, camera 193, and wireless communication module 160, etc.

[0058] The wireless communication function of electronic device 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0059] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0060] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0061] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0062] The display screen 194 is used to display the display interface of an application, such as the display page of an application installed on the electronic device 100. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0063] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0064] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system and software code for at least one application program. The data storage area may store data generated during the use of electronic device 100 (e.g., captured images, recorded videos, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0065] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, images, videos, and other files can be saved on the external memory card.

[0066] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0067] The sensor module 180 may include a pressure sensor 180A, an acceleration sensor 180B, a touch sensor 180C, etc.

[0068] The pressure sensor 180A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A may be disposed on the display screen 194.

[0069] Touch sensor 180C, also known as a "touch panel," can be located on display screen 194. The touch sensor 180C and display screen 194 together form a touchscreen, also known as a "touch screen." Touch sensor 180C detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180C may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0070] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch buttons. Electronic device 100 can receive button inputs and generate key signal inputs related to user settings and function control. Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, touch operations applied to different applications (such as taking photos, audio playback, etc.) can correspond to different vibration feedback effects. Touch vibration feedback effects can also be customized. Indicator 192 can be an indicator light, used to indicate charging status, battery level changes, or to indicate messages, missed calls, notifications, etc. SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with electronic device 100.

[0071] It is understood that the components shown in Figure 2 do not constitute a specific limitation on the electronic device 100. The electronic device may include more or fewer components than shown, or combine some components, or separate some components, or have different component arrangements. Furthermore, the combination / connection relationships between the components in Figure 2 can also be adjusted and modified.

[0072] Figure 3 is a software structure block diagram of an electronic device provided in an embodiment of this application. As shown in Figure 3, the software structure of the electronic device can be a layered architecture. For example, the software can be divided into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the operating system is divided into four layers, from top to bottom: the application layer, the application framework layer (framework, FWK), the runtime and system libraries, and the kernel layer.

[0073] The application layer may include a series of application packages. As shown in Figure 3, the application layer may include a camera, settings, skin modules, user interface (UI), third-party applications, etc. Third-party applications may include gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, SMS, etc. In this embodiment, the application layer may include a target installation package of a target application that the electronic device requests to download from a server. The function files and layout files in this target installation package are adapted to the electronic device.

[0074] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer can include some predefined functions. As shown in Figure 3, the application framework layer can include a window manager, content provider, view system, phone manager, resource manager, and notification manager.

[0075] The window manager is used to manage windowed applications. It can obtain the screen size, determine if a status bar is present, lock the screen, and capture screenshots. The content provider stores and retrieves data, making this data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0076] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0077] A phone manager is used to provide communication functions for electronic devices. For example, it manages call status (including connection and disconnection).

[0078] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0079] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0080] The runtime includes the core libraries and the virtual machine. The runtime is responsible for the scheduling and management of the operating system.

[0081] The core library consists of two parts: one part contains the functionalities that the Java language needs to call, and the other part contains the core libraries of the operating system. The application layer and application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0082] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), image processing libraries, etc.

[0083] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0084] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0085] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0086] A 2D graphics engine is a graphics engine for 2D drawing.

[0087] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0088] The hardware layer can include various types of sensors, such as accelerometers, gyroscopes, and touch sensors.

[0089] It should be noted that the structures shown in Figures 2 and 3 are merely examples of electronic devices provided in the embodiments of this application, and cannot be used to limit the electronic devices provided in the embodiments of this application. In specific implementations, electronic devices may have more or fewer devices or modules than those shown in Figures 2 or 3.

[0090] The emotion detection method provided in this application includes a training method for an emotion detection model and an emotion detection method based on the emotion detection model. The training method for the emotion detection model and the emotion detection method based on the emotion detection model provided in this application will be further described below:

[0091] I. Training Methods for Emotion Detection Models

[0092] The training method for the emotion detection model provided in this application includes the steps of generating training sample data and training the emotion detection model. In the step of generating training sample data, the electronic device can collect the user's physiological data and obtain the emotion label corresponding to the physiological data. The electronic device can generate training sample data based on the user's physiological data and the corresponding emotion label. In the step of training the emotion detection model, the electronic device trains the initial model based on the training sample data to obtain a personalized target emotion detection model. The training method for the emotion detection model provided in this application will be further described below.

[0093] 1. Generate training sample data.

[0094] In this embodiment, the training sample set used to train the emotion detection model includes multiple training samples. Each training sample includes the user's physiological feature information and the corresponding emotion label. The user's physiological feature information is generated by the electronic device based on the acquired user's physiological data. The emotion label corresponding to the physiological feature information can be set by the user, generated by a general emotion detection model, calibrated by the user after generation by a general emotion detection model, or calibrated by a historical emotion detection model. The historical emotion detection model includes at least one emotion detection model before the target emotion detection model was updated. The target emotion detection model is the emotion detection model currently used to detect the user's emotions, trained based on the user's physiological feature information and emotion label.

[0095] Optionally, the electronic device can collect the user's physiological data. For example, if the electronic device is a wearable device, it can collect the user's physiological data while the user is wearing it; or the electronic device can receive physiological data collected by other electronic devices, such as a mobile phone, which can receive physiological data collected by a wearable device. In this embodiment, the physiological data acquired by the electronic device can be the user's heart rate interval (RR interval, RRI).

[0096] In this embodiment of the application, the electronic device can generate physiological feature information based on the user's physiological data. The physiological feature information can be used to input into the emotion detection model to obtain the user's emotional result. The electronic device can also store the physiological feature information and its corresponding emotion label as training samples.

[0097] Taking the physiological data of the user collected by the electronic device in this application embodiment as RRI as an example, the physiological feature information corresponding to RRI can be the heart rate variability (HRV) feature calculated based on RRI. For example, HRV feature can include at least one of the following parameters: mean RR interval, standard deviation of RR interval, standard deviation of adjacent RRI, median of RRI, proportion of RRI intervals with a duration of 50ms, and proportion of RRI intervals with a duration of 20ms.

[0098] For example, the mean RR interval can satisfy the following relationship:

[0099] in, Represents the mean of the RR interval, RR n This represents the time interval between the (N+1)th heartbeat and the Nth heartbeat, where n is a positive integer less than or equal to N, and N is a positive integer greater than 1.

[0100] The standard deviation of the RR interval (SDNN) can satisfy the following relationship:

[0101] The standard deviations SD of adjacent RRIs can satisfy the following relationship:

[0102] Where, ΔRR m This represents the difference between the (m+1)th RR interval and the mth RR interval. This indicates the calculation of M ΔRR values. 2 The average value of M ΔRR 2 include E{ΔRR M} represents calculating the average of M ΔRRs, where the M ΔRRs include ΔRR1 to ΔRR2. MWhere m is a positive integer less than or equal to M, and M is a positive integer greater than 1, and M can take the value N-1.

[0103] When N is odd, the median RR interval m 0.5 The following relationship can be satisfied: m 0.5 =RR (N+1) / 2

[0104] When N is even, the median RR interval m 0.5 The following relationship can be satisfied:

[0105] The RRI interval 50ms ratio pNN50 can satisfy the following relationship:

[0106] NN50 represents the number of intervals where the difference between adjacent RR intervals is greater than 50 ms.

[0107] The RRI interval 20ms proportion pNN20 can satisfy the following relationship:

[0108] NN20 represents the number of intervals where the difference between adjacent RR intervals is greater than 20 ms.

[0109] In this embodiment, since different users have different physiological characteristics, the baseline values ​​and fluctuation ranges of the characteristic indicators of different people are different. Therefore, in this embodiment, the electronic device can use the average value and standard deviation of the HRV characteristics of the historically acquired RRI as the feature baseline, and normalize the HRV characteristics calculated above based on the feature baseline. The normalized physiological characteristic information can ensure that the numerical differences and fluctuations between different features are within the same order of magnitude and within a controllable range, thereby improving the robustness and generalization ability of the emotion detection model.

[0110] For example, taking the normalization of the standard deviation of the RR interval as an example, electronic devices can calculate the SDNN mean and the SDNN standard deviation based on the historically acquired RRI. The SDNN mean and the SDNN standard deviation satisfy the following relationship:

[0111] in, This represents the average value of the SDNN. N This represents the SDNN features calculated from the historical RRI data, and σSDNN represents the standard deviation of the SDNN.

[0112] After calculating the SDNN mean and standard deviation, electronic devices can use these as the baseline for SDNN features and normalize them. For example, the normalized SDNN can satisfy the following relationship:

[0113] Normalization

[0114] It is understood that in the above embodiments, the electronic device can calculate HRV features based on the acquired RRI. HRV features may include at least one of the following parameters: RR interval mean, RR interval standard deviation, standard deviation of adjacent RRIs, RRI median, RRI interval proportion of 50ms and RRI interval proportion of 20ms. The normalization method of the parameters such as RR interval mean, standard deviation of adjacent RRIs, RRI median, RRI interval proportion of 50ms and RRI interval proportion can refer to the normalization method of RR interval standard deviation SDNN mentioned above, and the repeated parts will not be repeated.

[0115] Electronic devices can use normalized HRV features as the user's physiological feature information and obtain the emotion label corresponding to this physiological feature information. Optionally, after obtaining the user's physiological feature information, the electronic device can input the user's physiological feature information into a general emotion detection model and obtain the emotion result output by the general emotion detection model. The electronic device can use the emotion result output by the general emotion detection model as the emotion label corresponding to the physiological feature information, or the electronic device can display the detected emotion result on the screen, and after the user calibrates the emotion result, the electronic device can use the calibrated emotion result as the emotion label corresponding to the physiological feature information.

[0116] For example, after obtaining the user's emotional result based on the user's physiological characteristics and a general emotion detection model, the electronic device can display the user's emotional result to the user. The user can view the emotional result and calibrate it. For example, Figure 4 is a schematic diagram of an emotional result display interface of an electronic device provided in an embodiment of this application. Referring to Figure 4, the electronic device may have an emotion detection application installed, or the health application of the electronic device may include an emotion detection function. After the electronic device performs emotion detection on the user, it can display the obtained emotional result in the emotion interface. As shown in Figure 4, interface A may include emotional results detected at multiple historical moments. The emotional result detected by the electronic device in Figure 4 may be any one of "pleasure," "calm," and "unpleasant." Alternatively, the electronic device may display the most recently detected emotional result in the emotion interface, such as interface B, which shows an emotional detection result of "pleasure" 5 minutes ago when the user clicked to view the emotion interface. Users can modify the emotion results in either interface A or interface B. For example, if a user clicks the edit control for the emotion result corresponding to 16:00 in interface A, or clicks the edit control in interface B, the electronic device can jump to and display interface C. Interface C can include multiple candidate emotions, from which the user can select an emotion. The electronic device will then use the selected emotion as the calibrated emotion result. For instance, if interface C displays the emotion result detected by the model as "pleasure," and the user selects "calm" in interface C, the electronic device will change the emotion result corresponding to 16:00 from "pleasure" to "calm." Optionally, referring to interface A in Figure 4, users can also trigger batch modification of emotion results corresponding to multiple times in interface A. Specifically, users can select multiple emotion results that need to be modified uniformly in interface A, and then select an emotion in interface C displayed by the electronic device, thus allowing the electronic device to modify the multiple emotion results selected by the user uniformly. Optionally, as shown in interface D of Figure 4, the interface for displaying historical emotions on the electronic device can have other display styles, and the historical emotion interface can also include "association controls." Users can click on these "association controls," and the electronic device can then jump to interface E. In interface E, users can select events associated with the emotion result, such as "work," "study," "housework," or "fitness." When the electronic device displays interface D, users can click on the line corresponding to the emotion result at a specific moment in interface D. For example, if a user clicks on any position on the line corresponding to the emotion result "pleasure" at 16:00, the electronic device can jump to interface E, allowing the user to view or modify the events associated with that emotion result in interface E.In some examples, electronic devices can trigger emotion detection when a user's state meets preset conditions. These preset conditions might include determining if the user is in motion, working, or eating. The electronic device can then directly associate the detected emotion with the event corresponding to the user's detected state. Through this design, the electronic device can store user-selected events or events determined by the device as events associated with the emotion result, thereby enriching the relevant information about the emotion result.

[0117] For example, Figure 5 is a schematic diagram of an electronic device displaying emotion results according to an embodiment of this application. Referring to interface A in Figure 5, after the electronic device completes a day's emotion detection for the user, it can display a mood pop-up window "Today's Mood" on the screen at a preset time. This mood pop-up window is used to display the user's emotion results for the day. When generating "Today's Mood," the electronic device can use the emotion with the highest proportion among the emotion results detected during the day that is greater than a preset proportion as "Today's Mood." The user can trigger emotion editing in interface A. For example, after the user triggers emotion editing in interface A, the electronic device can jump to display interface A as shown in Figure 4. The user can select the time corresponding to one or more emotions that they want to modify. At this time, the electronic device can jump to display interface C as shown in Figure 5. Interface C can include multiple candidate emotions. The user can select an emotion from the multiple candidate emotions, and the electronic device uses the emotion selected by the user as the calibrated emotion result.

[0118] It should be noted that the above embodiments are exemplified by the emotional result detected by the electronic device being any one of "pleasure", "calm" and "unpleasant". However, the above emotional results are only examples and not limitations. Other emotional results may occur in specific implementations, and this application embodiment does not limit them.

[0119] Optionally, after the electronic device detects the user's emotion, it can display a mood pop-up on the screen in real time. Referring to interface B in Figure 5, the mood pop-up is used to display the most recent emotion result. The user can trigger the editing of the emotion in interface B. Similar to the embodiment shown in Figure 4, when the user triggers the editing of the emotion result, the electronic device can jump to display interface C. Interface C can include multiple candidate emotions. The user can select an emotion from the multiple candidate emotions, and the electronic device uses the emotion selected by the user as the calibrated emotion result.

[0120] In this embodiment, if the user modifies the emotion result detected by the electronic device, the electronic device will use the modified emotion result as the emotion label corresponding to the physiological feature information; if the user does not modify the emotion result detected by the electronic device, the electronic device will still retain the emotion result detected by the electronic device as the emotion label corresponding to the physiological feature information.

[0121] Furthermore, in the emotion detection method provided in this application embodiment, after the electronic device completes the training of the target emotion detection model, the user can trigger the recalibration of historical emotions using the target emotion detection model. The emotion results stored by the electronic device then include the calibrated emotion results of the target emotion detection model. When generating training samples for model update training, the electronic device can also use the calibrated emotion results of the target emotion detection model as emotion labels in the training samples, which can be used to update the target emotion detection model. Similarly, when training the target emotion detection model, the emotion labels in the training samples can also include calibrated emotion results of historical emotion detection models, which include at least one emotion detection model used before the target emotion detection model was used during the emotion detection process. For example, an electronic device trains an initial model to obtain a target emotion detection model 0, and uses target emotion detection model 0 to detect the user's emotions; the electronic device updates and trains the target emotion detection model to obtain a target emotion detection model 1, and uses target emotion detection model 1 to replace target emotion detection model 0, so as to use target emotion detection model 1 to detect the user's emotions; the electronic device updates and trains target emotion detection model 1 to obtain a target emotion detection model 2, and uses target emotion detection model 2 to replace target emotion detection model 1, so as to use target emotion detection model 2 to detect the user's emotions. In this example, when the electronic device updates and trains target emotion detection model 1 to obtain target emotion detection model 2, the emotion labels in the training samples used by the electronic device may include the emotion results calibrated by target emotion detection model 1, and / or, the emotion results calibrated by target emotion detection model 0.

[0122] It should be noted that if the user has manually calibrated the emotion result, the electronic device will no longer use the target emotion detection model to calibrate the emotion result. Optionally, when displaying emotion results corresponding to multiple time points, the electronic device can display the emotion result modified by the user and the emotion result not modified by the user in different display styles. Furthermore, when storing physiological feature information and emotion results corresponding to multiple time points, the electronic device can also store identification information to indicate whether the user has modified the emotion result.

[0123] It should be noted that the above description of how to obtain emotion labels corresponding to physiological feature information in training samples is only an example and not a limitation. In specific implementations, other methods can also be used to obtain emotion labels corresponding to physiological feature information, such as allowing users to input their emotions at the time of collection through a user interface after collecting users' physiological data.

[0124] 2. Training the emotion detection model

[0125] In this embodiment, the electronic device acquires training samples corresponding to multiple time points based on the training sample acquisition method described above. When the number of training samples in the training sample set is greater than a preset threshold, the electronic device can use the training sample set to train the initial model to be trained in order to obtain the target emotion detection model. The initial model to be trained can be a general emotion detection model, such as an extreme gradient boosting (XGBoost) decision tree model.

[0126] In one optional implementation, the electronic device can train the model using gradient descent. During model training, the electronic device can input physiological feature information from the training samples into the initial model and obtain the emotional result output by the initial model. The electronic device can calculate the loss value between the emotional result output by the initial model and the emotional labels in the training samples based on a preset loss function, and adjust the model parameters of the initial model according to the loss value. The above process is repeated until the loss value converges, at which point the electronic device determines that the model training is complete and obtains the target emotion detection model.

[0127] It should be noted that the above model training method is illustrated by using an electronic device to train an initial model to obtain a target emotion detection model. In specific implementations, the model training process can also be performed by the server shown in Figure 1 or other electronic devices associated with the electronic device. Specific implementations can refer to the above description, and repeated details will not be elaborated upon. Furthermore, the above model training method can be used to train a target emotion detection model, a general emotion detection model, or to update and train a target emotion detection model. The above model training method is merely an example and not a limitation; other model training methods can also be used in specific implementations, and this application embodiment does not limit this approach.

[0128] In some embodiments, before the electronic device completes the training of the first target emotion detection model, it can use a general emotion detection model for emotion detection. After completing the training of the first target emotion detection model, the electronic device can replace the general emotion detection model with the target emotion detection model to perform emotion detection on the user. Alternatively, the electronic device can combine the general emotion detection model and the target emotion detection model to perform emotion detection on the user. To ensure the accuracy of the general emotion detection model, the electronic device can also update the general emotion detection model. Optionally, the electronic device can obtain training data from a server for updating and training the general emotion detection model. This training data may include physiological data and corresponding emotion tags authorized and uploaded by multiple users. The electronic device can process the training data into training samples that can be used for model training according to the training sample acquisition method provided in the above embodiments, and train the general emotion detection model based on the training samples to complete the online update of the general emotion detection model. In specific implementations, the method of the electronic device training the general emotion detection model can refer to the method of the electronic device training the initial model in the above embodiments, and the repeated parts will not be described again. It is understandable that updating a general emotion detection model with a large amount of training data uploaded by different users can improve the emotion detection capability of the general emotion detection model and enhance its universality.

[0129] II. Emotion Detection Methods Based on Emotion Detection Models

[0130] In this embodiment, the electronic device can detect the user's emotions according to a preset cycle, such as acquiring the user's physiological data and detecting the user's emotions every 10 minutes; or the user can trigger the electronic device to detect the user's emotions, in which case the electronic device responds to the user's operation, acquires the user's physiological data, and detects the user's emotions; or the electronic device can acquire the user's physiological data and detect the user's emotions when the user's state meets preset conditions, such as conditions for determining that the user is in a state of exercise, conditions for determining that the user is in a state of work, conditions for determining that the user is in a state of eating, etc. After acquiring the user's physiological data, the electronic device can process the physiological data based on the method of generating physiological feature information provided in the above embodiments, including calculating HRV features, normalization processing, etc. The electronic device can input the processed physiological feature information into the emotion detection model and obtain the emotion result output by the emotion detection model.

[0131] In one alternative implementation, before the electronic device has completed training the target emotion detection model, it can detect the user's emotions based on a general emotion detection model. After training the target emotion detection model, the electronic device can compare the accuracy and specificity of the general emotion detection model and the target emotion detection model. If the target emotion detection model is determined to perform better, it is used to replace the general emotion detection model. When the electronic device needs to detect the user's emotions, it uses the target emotion detection model.

[0132] For example, Figure 6 is a flowchart of an emotion detection method provided in an embodiment of this application. Referring to Figure 6, the electronic device can perform emotion detection on the user simultaneously with the training of the target emotion detection model. When the electronic device determines that emotion detection on the user is necessary, it acquires the user's current RRI data and calculates HRV feature information based on the RRI data. The electronic device calculates historical HRV feature information based on historically acquired user RRI data and updates the RRI feature baseline based on the historically acquired RRI data. The electronic device normalizes the HRV feature information corresponding to the currently acquired RRI data according to the RRI feature baseline to obtain physiological feature information. The electronic device inputs the physiological feature information into a general emotion detection model and obtains the emotion result output by the general emotion detection model. If the electronic device has not yet completed the training of the target emotion detection model, it displays the emotion result output by the general emotion detection model to the user.

[0133] Referring to Figure 6, the electronic device normalizes historical HRV feature information based on the RRI feature baseline to obtain physiological feature information in the training samples. The electronic device uses the emotion results corresponding to the historically acquired RRI data as emotion labels, thus obtaining multiple training samples. Each training sample includes physiological feature information and the corresponding emotion label. The electronic device trains the initial model based on the training samples to obtain the target emotion detection model. The electronic device can compare the performance of the general emotion detection model and the target emotion detection model, such as comparing the accuracy and specificity of the general emotion detection model and the target emotion detection model. When it is determined that the target emotion detection model performs better, the target emotion detection model is used to detect the user's emotion. For example, an electronic device can select at least one training sample from a training sample set, input the physiological feature information from this training sample into a general emotion detection model and a target emotion detection model, respectively, and obtain the emotion result output by the general emotion detection model. Based on the emotion result output by the general emotion detection model and the emotion labels in the training samples, the electronic device calculates the accuracy and specificity of the general emotion detection model. Similarly, it obtains the emotion result output by the target emotion detection model, calculates the accuracy and specificity of the target emotion detection model based on the emotion result output by the target emotion detection model and the emotion labels in the training samples. When the accuracy and specificity of the target emotion detection model are higher than those of the general emotion detection model, the electronic device can determine that the target emotion detection model performs better. After replacing the general emotion detection model with the target emotion detection model, when the electronic device determines that emotion detection of the user is required, it inputs the user's physiological feature information into the target emotion detection model, obtains the emotion result output by the target emotion detection model, and displays this emotion result to the user.

[0134] In another alternative implementation, the electronic device can also perform emotion detection based on both a general emotion detection model and a target emotion detection model, and fuse the emotion results output by the two models to obtain the final emotion result.

[0135] For example, Figure 7 is a flowchart of an emotion detection method provided in an embodiment of this application. Referring to Figure 7, when the electronic device determines that emotion detection of a user needs to be performed, it acquires the user's physiological characteristic information. The electronic device can input the physiological characteristic information into a general emotion detection model and a target emotion detection model respectively. The electronic device acquires the probability values ​​corresponding to each emotion output by the general emotion detection model and the probability values ​​corresponding to each emotion output by the target emotion detection model. The electronic device performs probability averaging on the probability values ​​corresponding to each emotion output by the general emotion detection model and the probability values ​​corresponding to each emotion output by the target emotion detection model, thereby calculating the average probability value for each emotion, and taking the emotion result with the largest average probability value as the final detected emotion result.

[0136] It should be noted that in the emotion detection method shown in Figure 7, the general emotion detection model and the target emotion detection model output probability values ​​for each emotion, such as the probability values ​​for "calm," "pleasure," and "unpleasantness." These probability values ​​can serve as an intermediate result for the general and target emotion detection models. In practice, the electronic device can obtain the probability values ​​for each emotion from both models and determine the final detected emotion result based on these values. Furthermore, as described in the preceding embodiments, the electronic device can also use either the target emotion detection model or the general emotion detection model alone for emotion detection. In this case, the electronic device can directly obtain the final emotion result output by either the target or general emotion detection model.

[0137] In the emotion detection method provided in this application embodiment, the electronic device can also update the target emotion detection model based on the collected user physiological data. For example, the electronic device can store the collected user physiological data and the corresponding emotion results, and the emotion results corresponding to the physiological data include the user-calibrated emotion results and / or the emotion results calibrated by historical emotion detection models. The historical emotion detection models include at least one emotion detection model used before the target emotion detection model was used during the emotion detection process. When the number of stored physiological data and emotion results exceeds a preset threshold, the electronic device can generate training samples based on the user's physiological data and emotion results to retrain and update the target emotion detection model, so that the target emotion detection model is more in line with the user's personalized characteristics and improves the detection accuracy of the target emotion detection model.

[0138] Optionally, after a user views and modifies the emotion result, the electronic device can also display a reminder message asking the user whether to update the model based on the modified emotion result. After the user confirms, the electronic device can generate training samples based on the user's modified emotion result and train and update the target emotion detection model based on the training sample set including the training samples, thereby providing a way for the user to actively trigger the model update and improving the user experience.

[0139] For example, Figure 8 is a schematic diagram of an interface for updating an emotion detection model according to an embodiment of this application. Referring to Figure 8, after the user edits the emotion result in interface A displayed on the electronic device, the electronic device can jump to display interface B. Interface B includes a reminder message, such as "Update the emotion detection model with the current calibration data?". After the user triggers a confirmation operation in interface B, the electronic device can generate training samples based on the user's modified emotion result, and train and update the target emotion detection model based on the training sample set including the training samples.

[0140] In this embodiment, after the electronic device completes the training of the target emotion detection model, or after updating the target emotion detection model, the electronic device can also ask the user whether to calibrate the emotion results before the update of the target emotion detection model. After the user confirms the calibration, the electronic device can input historical physiological feature information into the updated target emotion detection model and obtain the emotion results output by the updated target emotion detection model. The electronic device can update the historical emotion results based on the emotion results output by the target emotion detection model to complete the calibration of historical data. Optionally, when calibrating historical emotion results, the electronic device only calibrates emotion results that have not been modified by the user. If the user has modified the emotion results, the calibration of those emotion results is skipped. That is, the priority of the emotion results modified by the user is higher than the emotion results detected by the model.

[0141] For example, Figure 9 is a schematic diagram of an interface that asks the user whether to calibrate historical emotion results according to an embodiment of this application. Referring to Figure 9, after the electronic device determines that the training of the target emotion detection model has been completed, or after the electronic device updates the target emotion detection model, it can display the interface A shown in Figure 9. The interface A includes a reminder message, such as "The emotion detection model has been updated. Do you want to use the model to refresh the historical uncalibrated emotions?" After the user triggers the confirmation operation, the electronic device can calibrate the emotion results that the user has not modified in the historical emotion results based on the updated target emotion detection model. Optionally, after the electronic device has completed training of the target emotion detection model or updated the target emotion detection model, it can also display interface B as shown in Figure 9. Interface B contains a reminder message asking the user whether to use the updated target emotion detection model to calibrate uncalibrated emotions. Interface B also includes a "Select Emotion" control. After the user clicks this control, the electronic device can jump to interface C, which includes multiple uncalibrated emotion results. The user can select the emotion that needs to be calibrated using the updated target emotion detection model. After the user completes the selection, the electronic device can calibrate the selected emotion result based on the updated target emotion detection model. Optionally, the electronic device can also calibrate unmodified emotion results from historical emotion results within a first time period based on the updated target emotion detection model. This first time period can be, for example, a preset time period before the target emotion detection model was updated, such as 24 hours before the update, or a time period selected by the user.

[0142] For example, referring to interface D in Figure 9, the historical emotion results interface displayed by the electronic device can also include whether the emotion results corresponding to each time point have been calibrated. As shown in interface D, the electronic device can display a "manually calibrated" mark at the emotion results that the user has manually calibrated. For example, in the emotion results displayed in interface D, "pleasure" at 16:00 and "calm" at 16:10 are emotion results that the user has manually calibrated, while the uncalibrated emotion results include "calm" at 16:20 and "calm" at 16:30. After calibrating the uncalibrated emotion results based on the updated target emotion detection model, referring to interface E in Figure 9, the electronic device can display a "model calibrated" mark at the emotion results that the model has calibrated. For example, in interface E, the calibrated emotion result at 16:20 changes from "calm" to "pleasure," while the calibrated emotion result at 16:30 remains unchanged. It should be noted that the "manually calibrated" and "model calibrated" marks in interfaces D and E shown in Figure 9 are only examples and not limitations. In specific implementations, the electronic device can also mark the calibrated emotion results with other marks, and this application embodiment does not limit this.

[0143] Optionally, in this embodiment, the use and training of the target emotion detection model can be carried out simultaneously, thereby optimizing the model performance through continuous iterative training. When the electronic device updates and trains the target emotion detection model, the training samples used are the user's physiological feature information and the emotion labels corresponding to the physiological feature information acquired by the electronic device in the past. After the electronic device updates the target emotion detection model, the electronic device can calibrate the emotion labels in the training samples based on the updated target emotion detection model. That is, after the electronic device completes one update training of the target emotion detection model, it can use the updated target emotion detection model to update the training samples with uncalibrated emotions in the training sample set, thereby continuously improving the accuracy of the training samples, and further improving the detection accuracy of the target emotion detection model obtained based on the training samples.

[0144] Based on the same inventive concept, this application also provides an electronic device that can be used to execute the emotion detection method described in the above embodiments. Figure 10 is a schematic diagram of the structure of an electronic device provided in this application. Referring to Figure 10, the electronic device may include a feature extraction unit 1001, an emotion data storage unit 1002, an emotion detection model 1003, and an emotion data calibration unit 1004.

[0145] The feature extraction unit 1001 is used to acquire the user's physiological data and determine physiological feature information based on the user's physiological data.

[0146] The emotion data storage unit 1002 is used to store the user's physiological data, physiological characteristic information, and emotional results acquired by the electronic device.

[0147] The emotion detection model 1003 is used to detect a user's emotions based on the user's physiological characteristics and generate emotion results. In this embodiment, the emotion detection model 1003 may include a general emotion detection model and a target emotion detection model.

[0148] The emotion data calibration unit 1004 provides an interface for users to calibrate emotion results, and users can modify the emotion results in this interface.

[0149] The following is a further description of an emotion detection method provided by an embodiment of this application. Figure 11 is a flowchart of an emotion detection method provided by an embodiment of this application. Referring to Figure 11, the method includes the following steps:

[0150] S1101: The electronic device acquires the user's first physiological data at the first moment.

[0151] S1102: The electronic device generates first physiological characteristic information based on the user's first physiological data.

[0152] Optionally, S1101-S1102 can be performed by the feature extraction unit 1001 of the electronic device.

[0153] S1103: The electronic device determines the first emotion result based on the first physiological characteristic information and a general emotion detection model.

[0154] S1104: The electronic device displays the first emotion result.

[0155] S1105: In response to the user's first operation, the electronic device changes the emotional result corresponding to the first moment from the first emotional result to the second emotional result.

[0156] The first operation is a user-triggered operation to modify the emotion result. Optionally, the interface displaying the first emotion result in S1104 may also include an emotion result editing control, and the user-triggered first operation may be a click operation on the emotion result editing control. For example, the first operation may be a user-triggered operation to modify the emotion result in interface A or interface B shown in Figure 4.

[0157] Optionally, S1105 can be performed by the emotion data calibration unit 1004 of the electronic device.

[0158] S1106: The electronic device stores the first physiological feature information and the second emotional result as a training sample.

[0159] Optionally, S1106 can be executed by the emotion data storage unit 1002 of the electronic device.

[0160] S1107: The electronic device determines that the number of training samples is greater than a preset threshold, and trains the initial model based on multiple training samples to obtain the target emotion detection model.

[0161] S1108: Electronic devices should replace general emotion detection models with targeted emotion detection models.

[0162] Optionally, before S1108, the electronic device can also compare the performance of the general emotion detection model and the target emotion detection model. If it is determined that the performance of the target emotion detection model is superior to that of the general emotion detection model, then S1108 is executed. The method by which the electronic device compares the performance of the general emotion detection model and the target emotion detection model can be found in the above embodiments, and will not be repeated here.

[0163] S1109: In response to the user's second action, the electronic device calibrates historical emotions based on the target emotion detection model and generates multiple training samples based on the calibrated emotion results.

[0164] The second operation is a user-triggered operation that calibrates historical emotions using the target emotion detection model. Optionally, before S1109, the electronic device may display a second reminder message on the screen. The second reminder message asks the user whether to calibrate uncalibrated emotion results among the multiple emotion results corresponding to multiple moments stored in the electronic device based on the updated target emotion detection model. The user-triggered second operation can be a click operation on the confirmation control corresponding to the second reminder message. For example, the second operation can be the user clicking the "Confirm" control in the interface shown in Figure 9.

[0165] S1110: The electronic device retrains the target emotion detection model based on the training sample set to obtain an updated target emotion detection model.

[0166] It should be noted that the training sample set used by the electronic device in S1110 includes multiple training samples generated by the electronic device in S1109 based on the calibrated emotion results. The training sample set may also include at least one of the following: training samples generated based on the user-calibrated emotion results, training samples generated based on the emotion results detected by the general emotion detection model, and training samples generated based on the emotion results detected by the historical emotion detection model.

[0167] S1111: The electronic device acquires the user's second physiological data at a second moment.

[0168] S1112: The electronic device generates second physiological characteristic information based on the user's second physiological data.

[0169] Optionally, S1111-S1112 can be performed by the feature extraction unit 1001 of the electronic device.

[0170] S1113: The electronic device determines the third emotion result based on the updated target emotion detection model according to the second physiological characteristic information.

[0171] S1114: Electronic devices display third emotional results.

[0172] It should be noted that the implementation of S1110 in this application does not limit the timing of execution. The electronic device can execute S1110 after executing S1109, or the electronic device can execute S1110 after storing more training samples.

[0173] Furthermore, the embodiment shown in Figure 11 is only an example and not a limitation. In practice, the emotion detection method may include more or fewer steps based on the scheme provided in the foregoing embodiments of this application. For specific implementation, please refer to the foregoing embodiments, and repeated details will not be repeated.

[0174] This application also provides an electronic device. As shown in FIG12, the electronic device 1200 includes: a bus 1202, a processor 1204, a memory 1206, and a communication interface 1208. The processor 1204, the memory 1206, and the communication interface 1208 communicate with each other via the bus 1202. It should be understood that this application does not limit the number of processors and memories in the electronic device 1200.

[0175] Bus 1202 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 12, but this does not imply that there is only one bus or one type of bus. Bus 1202 can include pathways for transmitting information between various components of electronic device 1200 (e.g., memory 1206, processor 1204, communication interface 1208).

[0176] The processor 1204 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0177] The memory 1206 may include volatile memory, such as random access memory (RAM). The processor 1204 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0178] The memory 1206 stores executable program code, and the processor 1204 executes the executable program code to implement the functions executed by the virtual machine in the aforementioned embodiments, thereby implementing the memory allocation method provided in this application embodiment. That is, the memory 1206 stores instructions for executing the memory allocation method.

[0179] Alternatively, the memory 1206 stores executable program code, and the processor 1204 executes the executable program code to implement the functions executed by the virtual machine in the embodiments of this application, thereby realizing the memory allocation method provided in the embodiments of this application.

[0180] The communication interface 1208 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable the virtual machine to receive or send data.

[0181] Based on the same inventive concept, this application also provides an emotion detection method, which can be executed by an electronic device, such as a wearable device. Figure 13 is a flowchart illustrating an emotion detection method provided in this application. Referring to Figure 13, the method includes the following steps:

[0182] S1301: The electronic device acquires the user's first physiological data at the first moment and generates first physiological characteristic information based on the first physiological data.

[0183] S1302: The electronic device generates a first emotion result corresponding to the first moment based on the first physiological feature information and the target emotion detection model; wherein, the target emotion detection model is trained based on the training sample set corresponding to the user's user data, the training sample set corresponding to the user data is generated based on the user's physiological data and the emotion result corresponding to the physiological data, the emotion result corresponding to the physiological data includes the emotion result calibrated by the user, and / or, the emotion result calibrated by the historical emotion detection model, the historical emotion detection model includes at least one emotion detection model before using the target emotion detection model during the emotion detection process.

[0184] It should be noted that the emotion detection method shown in Figure 13 of this application can be referred to the above embodiments of this application in specific implementation, and repeated parts will not be described again.

[0185] Based on the above embodiments, this application also provides a computer program product containing instructions, which, when run on a computer, causes the computer to execute the methods described in the embodiments of this application.

[0186] Based on the above embodiments, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a computer, causes the computer to perform the methods described in the embodiments of this application.

[0187] Based on the above embodiments, this application also provides a chip for reading computer programs stored in a memory to implement the methods described in the embodiments of this application.

[0188] Based on the above embodiments, this application provides a chip system including a processor for supporting a computer device in implementing the methods described in the embodiments of this application. In one possible design, the chip system further includes a memory for storing necessary programs and data of the computer device. This chip system may be composed of chips or may include chips and other discrete devices.

[0189] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0190] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0191] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0192] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0193] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of protection of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. An emotion detection method, characterized in that, Applied to electronic devices, the method includes: Acquire the user's first physiological data at the first moment, and generate first physiological feature information based on the first physiological data; Based on the first physiological feature information, a first emotion result corresponding to the first moment is generated based on the target emotion detection model; wherein, the target emotion detection model is trained based on the training sample set corresponding to the user's user data, the training sample set corresponding to the user data is generated based on the user's physiological data and the emotion result corresponding to the physiological data, the emotion result corresponding to the physiological data includes the user's calibrated emotion result, and / or, the emotion result calibrated by the historical emotion detection model, the historical emotion detection model includes at least one emotion detection model used before the target emotion detection model was used during the emotion detection process.

2. The method as described in claim 1, characterized in that, After generating the first emotion result corresponding to the first moment based on the first physiological feature information and the target emotion detection model, the method further includes: Displays the first emotion result corresponding to the first moment; In response to the user's first action, the emotion result editing interface is displayed; In response to a second operation performed by the user on the emotion result editing interface, the emotion result corresponding to the first moment is changed from the first emotion result to the second emotion result.

3. The method as described in claim 1 or 2, characterized in that, The method further includes: Display a first reminder message, which asks the user whether to update the target emotion detection model based on the user's calibrated emotion results. In response to the user's third operation, a first training sample is generated based on the first physiological data and the second emotional result. The target emotion detection model is updated and trained based on the training sample set including the first training sample to obtain an updated target emotion detection model. The updated target emotion detection model is then used to replace the target emotion detection model.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: Once it is determined that the target emotion detection model has been updated, a second reminder message is displayed. The second reminder message is used to ask the user whether to calibrate the uncalibrated emotion results among the multiple emotion results corresponding to multiple times stored in the electronic device based on the updated target emotion detection model. In response to the fourth action triggered by the user, the uncalibrated emotion result is calibrated based on the updated target emotion detection model.

5. The method as described in claim 4, characterized in that, The uncalibrated emotion result is at least one emotion result selected by the user.

6. The method according to any one of claims 1-5, characterized in that, The step of generating the first emotion result corresponding to the first moment based on the first physiological feature information and the target emotion detection model includes: A third emotion result is generated based on the first physiological feature information and the target emotion detection model, and a fourth emotion result is generated based on the first physiological feature information and the general emotion detection model. The first emotional result is determined based on the third emotional result and the fourth emotional result.

7. The method as described in claim 6, characterized in that, The third emotional result includes a first probability value corresponding to each of the multiple emotions; the fourth emotional result includes a second probability value corresponding to each of the multiple emotions. Determining the first emotional result based on the third emotional result and the fourth emotional result includes: Calculate the average probability value of each emotion among the plurality of emotions based on the third emotion result and the fourth emotion result, and take the emotion with the highest average probability value among the plurality of emotions as the first emotion result.

8. The method according to any one of claims 1-7, characterized in that, The emotional results corresponding to the physiological data also include emotional results generated by a general emotion detection model, and / or emotional results generated by a historical emotion detection model.

9. The method according to any one of claims 1-8, characterized in that, The first physiological data is the heart rate interval (RRI); the step of generating first physiological feature information based on the first physiological data includes: Calculate the user's heart rate variability (HRV) characteristics based on the first physiological data; Calculate the user's characteristic baseline based on the user's historical physiological data; The HRV features are normalized based on the feature baseline to obtain the first physiological feature information.

10. The method as described in claim 9, characterized in that, The user's HRV characteristics include at least one of the following: mean RR interval, standard deviation of RR interval, standard deviation of adjacent RRIs, median RRI, proportion of RRI intervals with a duration of 50 ms, and proportion of RRI intervals with a duration of 20 ms.

11. An electronic device, characterized in that, It includes at least one processor coupled to at least one memory, the at least one processor being configured to read a program stored in the at least one memory to perform the method as described in any one of claims 1-10.

12. A readable storage medium, characterized in that, The readable storage medium stores instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-10.

13. A computer program product containing instructions, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1-10.

14. A chip, characterized in that, The chip is used to read a computer program stored in a memory to execute the method as described in any one of claims 1-10.

Images