Emotion adjustment system and emotion adjustment method

Abstract

The present application relates to an emotion adjustment system and an emotion adjustment method. An emotion adjustment system for determining a user's emotion based on the user's voice includes a microphone configured to receive the user's voice; a controller configured to extract a plurality of sound quality factors in response to processing the user's voice, calculate a depression index of the user based on at least one of the plurality of sound quality factors, identify a state of the user's emotion as a depression state when the depression index is above a preset value, determine the depression state as a first state or a second state based on a correlation between at least two of the plurality of sound quality factors, and transmit a control command corresponding to the state of the user's emotion identified as the first state or the second state; and a feedback device configured to perform an operation corresponding to the control command.

Description

Technical Field

[0001] This disclosure relates to an emotion adjustment system and an emotion adjustment method, and more specifically, to an emotion adjustment system and an emotion adjustment method that can more accurately determine a user's emotions based on the user's voice. Background Technology

[0002] Recently, research has been actively conducted on technologies for judging users' emotions using various sensors. Furthermore, research is also underway on technologies that can induce positive emotions in users through the use of various sensors.

[0003] For example, an emotion regulation system can provide useful services to users by judging their emotional state and providing several types of feedback devices that can perform actions that reflect their emotional state.

[0004] Examples of emotion regulation systems include AI speakers or various devices or vehicles equipped with AI speakers.

[0005] Specifically, vehicles can use various sensors such as biosignal sensors, microphones, and cameras to determine the user's emotional state and provide feedback that can alleviate or amplify the user's emotions by using various feedback devices such as vibrating elements set in seats, speakers, or air conditioners.

[0006] However, recent technologies require additional components such as biosignal sensors and cameras to determine the user's emotions, thus increasing costs. Furthermore, these technologies only determine whether the user's current emotional state in the vehicle is positive or negative. Moreover, based on the determined positive or negative emotional state, these technologies only provide feedback to adjust the output of components within the vehicle. Summary of the Invention

[0007] Therefore, one aspect of this disclosure provides an emotion adjustment system and method that can specifically identify a user's emotions based on the user's voice and provide feedback corresponding to the user's emotions.

[0008] According to one aspect of this disclosure, an emotion adjustment system includes: a microphone configured to receive a user's voice; a controller configured to extract multiple sound quality factors in response to processing the user's voice, calculate a user's depression index based on at least one of the multiple sound quality factors, identify the user's emotional state as a depressed state when the depression index is above a preset value, determine the depressed state as a first state or a second state based on the correlation between at least two of the multiple sound quality factors, and send a control command corresponding to the emotional state of the user identified as the first state or the second state; and a feedback device configured to perform an operation corresponding to the control command.

[0009] Various sound quality factors may include at least one of sound pressure level, loudness, sharpness, roughness, and wave intensity.

[0010] The controller can be configured to calculate a user’s depression index based on at least one of sound pressure level and loudness.

[0011] The controller can be configured to calculate a user’s depression index based on changes in sound pressure level or loudness.

[0012] The controller can be configured to calculate the noise index of a user’s speech based on the correlation between sharpness, roughness, and fluctuation intensity, and to classify the depressive state as either a first state or a second state based on the noise index.

[0013] The noise index can include a rattle index and a squeak index, and the controller can be configured to classify the depressive state as a first state or a second state based on the ratio of the rattle index to the squeak index.

[0014] When the loudness index is greater than the glide index, the controller can be configured to classify the depressed state as the first state, and when the glide index is greater than the loudness index, the controller can be configured to classify the depressed state as the second state.

[0015] The feedback device may include a vibrating element, and the controller may be configured to: when the user's emotional state is identified as a first state, send a first control command to cause the vibrating element to vibrate at a first frequency and a first intensity, and when the user's emotional state is identified as a second state, send a second control command to cause the vibrating element to vibrate at a second frequency and a second intensity; and the first frequency may be less than the second frequency and the first intensity may be less than the second intensity.

[0016] The feedback device may include a speaker, and the controller may be configured to: when the user's emotional state is identified as a first state, send a first control command to cause the speaker to play a first sound source, and when the user's emotional state is identified as a second state, send a second control command to cause the speaker to play a second sound source; and the frequency band of the first sound source may be narrower than the frequency band of the second sound source.

[0017] The controller can be configured to identify the user's emotional state as unstable or pleasant when the depression index is less than a preset value, and send a control command corresponding to the user's emotional state that is identified as unstable or pleasant.

[0018] According to one aspect of this disclosure, the emotion adjustment method includes: receiving a user's voice via a microphone; extracting multiple sound quality factors by processing the user's voice in response to a controller; calculating a depression index for the user based on at least one of the multiple sound quality factors; identifying the user's emotional state as a depressed state by the controller when the depression index is above a preset value; determining the depressed state as a first state or a second state by the controller based on the correlation between at least two of the multiple sound quality factors; sending a control command corresponding to the emotional state of the user identified as the first state or the second state by the controller; and executing the operation corresponding to the control command by a feedback device.

[0019] Various sound quality factors may include at least one of sound pressure level, loudness, sharpness, roughness, and wave intensity.

[0020] Calculating a user's depression index may include: calculating the user's depression index based on at least one of sound pressure level and loudness.

[0021] Calculating a user's depression index can include: calculating the user's depression index based on the pattern of change in sound pressure level or loudness.

[0022] Determining a depressive state as either a first or second state may include: calculating the noise index of the user's speech based on the correlation between sharpness, roughness, and fluctuation intensity; and determining the depressive state as either a first or second state based on the noise index.

[0023] The noise index can include a loudness index and a glide index, and classifying a depressive state as a first or second state based on the noise index can include classifying a depressive state as a first or second state based on the ratio of the loudness index to the glide index.

[0024] Determining a depressive state as a first or second state based on the ratio of the loudness index to the glissando index can include: when the loudness index is greater than the glissando index, the depressive state is determined to be a first state, and when the glissando index is greater than the loudness index, the depressive state is determined to be a second state.

[0025] The feedback device may include a vibrating element, and sending control commands corresponding to the emotional state of a user identified as a first state or a second state may include: when the user's emotional state is identified as a first state, sending a first control command to cause the vibrating element to vibrate at a first frequency and a first intensity; and when the user's emotional state is identified as a second state, sending a second control command to cause the vibrating element to vibrate at a second frequency and a second intensity; wherein the first frequency may be less than the second frequency and the first intensity may be less than the second intensity.

[0026] The feedback device may include a speaker, and sending control commands corresponding to the emotional state of a user identified as a first state or a second state may include: when the user's emotional state is identified as a first state, sending a first control command to cause the speaker to play a first sound source; and when the user's emotional state is identified as a second state, sending a second control command to cause the speaker to play a second sound source; and the frequency band of the first sound source may be narrower than the frequency band of the second sound source.

[0027] The emotion adjustment method may also include: when the depression index is less than a preset value, identifying the user's emotional state as an unstable state or a pleasant state; and sending a control command corresponding to the emotional state of the user identified as an unstable state or a pleasant state. Attached Figure Description

[0028] These and / or other aspects of this disclosure will become apparent and more readily understood from the following description of embodiments implemented in conjunction with the accompanying drawings, wherein:

[0029] Figure 1 This is a control block diagram of the emotion adjustment system according to the implementation method.

[0030] Figure 2 This is a flowchart of an emotion adjustment method according to the implementation method.

[0031] Figure 3 It is a diagram showing the relationships between various sound quality factors included in a user's speech.

[0032] Figure 4 This is a diagram showing the results of calculating the depression index based on the user's voice.

[0033] Figure 5 These are illustrations showing two embodiments of speech that are segmented into depressive moods.

[0034] Figure 6 This is a diagram illustrating a method for calculating the noise index based on the correlation between sound quality factors. Detailed Implementation

[0035] It should be understood that the term "vehicle" or "of a vehicle" or other similar terms used herein generally include motor vehicles, such as passenger cars including sport utility vehicles (SUVs), buses, trucks, various commercial vehicles, ships including various vessels, aircraft, etc., and include hybrid vehicles, electric vehicles, plug-in hybrid electric vehicles, hydrogen-powered vehicles, and other vehicles powered by alternative fuels (e.g., fuels derived from resources other than petroleum). As mentioned herein, a hybrid vehicle is a vehicle with two or more power sources, such as a vehicle that is both gasoline-powered and electric.

[0036] The terminology used herein is for descriptive purposes only and is not intended to be limiting of the invention. As used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It should be further understood that, when used herein, the terms “comprises” and / or “comprising” specify the presence of the stated feature, integral, step, operation, element, and / or component, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or combinations thereof. As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items. Throughout this specification, unless explicitly stated otherwise, the word “comprise” and variations such as “comprises” or “comprising” should be understood to imply the inclusion of the stated elements, but do not exclude any other elements. Furthermore, the terms “unit,” “device,” “worker,” and “module” described in this specification refer to a unit for performing at least one function and operation and that can be implemented by hardware components or software components and combinations thereof.

[0037] Furthermore, the control logic of this disclosure can encompass non-volatile computer-readable media located on a computer-readable medium, which contains executable program instructions that can be executed by a processor, controller, or the like. Examples of computer-readable media include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, flash memory drive, smart card, and optical data storage devices. The computer-readable medium can also be distributed across a network-coupled computer system, enabling it to be stored and executed in a distributed manner by, for example, a telematics server or a controller area network (CAN).

[0038] It should be understood that when an element is referred to as “connecting” another element, it can be directly or indirectly connected to the other element, wherein indirect connection includes “connection via wireless communication network”.

[0039] The terms first, second, etc. are used to distinguish one component from another, and the components are not limited by the terms described above.

[0040] Expressions used in the singular form cover expressions used in the plural form, unless they have a clearly distinct meaning in the context.

[0041] Reference numbers used in the operation are used for ease of description, and the reference numbers are not intended to describe the order of the operation, and the operation may be performed in a different order unless otherwise specified.

[0042] In the following description, embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0043] Figure 1 This is a control block diagram of the emotion adjustment system according to the implementation method.

[0044] According to the implementation method, the emotion adjustment system 1 can refer to artificial intelligence speakers, home appliances equipped with microphones, vehicles, etc.

[0045] In the following description, the case in which the emotion adjustment system 1 according to the implementation is provided as part of a vehicle is described as an example; however, the emotion adjustment system 1 is not limited to the implementation in a vehicle.

[0046] The emotion adjustment system 1 may include: a microphone 10 for receiving a user's voice; a controller 20 for recognizing the user's emotions in response to processing the user's voice and controlling a feedback device 30 by sending a control command corresponding to the user's emotional state; and the feedback device 30 for performing an operation corresponding to the control command sent from the controller 20.

[0047] Microphone 10 can receive the user's voice, perform processing such as amplification and noise reduction, and send the passenger's processed voice signal to controller 20.

[0048] When the emotion adjustment system 1 is a vehicle, the microphone 10 can be set to a location that is not limited to acquiring the voice signal of the user located inside the vehicle, and the number of microphones 10 is also unlimited.

[0049] The controller 20 may include: at least one memory 22 for storing various instructions and / or algorithms and / or programs and / or data for performing the operations described above and below; and at least one processor 21 for performing operations for controlling the various components of the emotion adjustment system 1 based on the various instructions and / or algorithms and / or programs and / or data stored in the memory 22.

[0050] The memory 22 can be implemented as at least one of a non-volatile memory device such as a cache, read-only memory (ROM), programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), and flash memory, or a volatile memory device such as random access memory (RAM), or a storage medium such as a hard disk drive (HDD) or CD-ROM, to store various types of information. However, this disclosure is not limited thereto.

[0051] The memory 22 can be implemented as a separate chip, or it can be implemented as a separate chip with the processor 21.

[0052] Feedback device 30 may include any device capable of providing feedback that evokes emotional changes in the user. For this purpose, feedback device 30 may provide various types of feedback, such as visual feedback, auditory feedback, tactile feedback, and olfactory feedback.

[0053] For example, the feedback device 30 may include a vibrating element 31 capable of providing tactile feedback to the user and a speaker 32 capable of providing auditory feedback, but is not limited thereto.

[0054] Specifically, although not shown in the accompanying drawings, the feedback device 30 may include various components such as air conditioning, lighting equipment, or a display.

[0055] When the mood adjustment system 1 is implemented in a vehicle, the vibration element 31 can be installed on the steering wheel and / or seat to provide tactile feedback to the user, and the speaker 32 can be installed in the vehicle to provide auditory feedback to the user.

[0056] The placement of the vibrating element 31 is unrestricted as long as it is positioned at one or more locations where the user can feel the vibration, and the placement of the speaker 32 is unrestricted as long as its position allows the user to hear the sound emitted by the speaker 32. Specifically, vibrating element 31 refers to one or more vibrating elements, and speaker 32 refers to one or more speakers.

[0057] Feedback device 30 can perform operations corresponding to control commands sent from controller 20.

[0058] In one implementation, the microphone 10, controller 20, and feedback device 30 can communicate with each other by performing Controller Area Network (CAN) communication to send corresponding information, and can also perform wired communication to send corresponding information. For example, when the mood adjustment system 1 is a vehicle used to control various electrical loads installed on the vehicle and to control communication between various electrical loads in the vehicle, a communication network including a body network, a multimedia network, and a chassis network is configured, and the controller 20 can connect the separate networks to exchange CAN communication messages.

[0059] As described above, the configurations of the emotion adjustment system 1, including its implementation, operation, and structure, according to various configurations, have been described. Hereinafter, emotion adjustment methods using various configurations of the emotion adjustment system 1 will be described in detail.

[0060] Figure 2 This is a flowchart of an emotion adjustment method according to the implementation method. Figure 3 It is a diagram showing the correlation between various sound quality factors included in a user's speech. Figure 4 This is a diagram showing the results of calculating the depression index based on the user's voice. Figure 5 These are illustrations showing two embodiments of speech that are segmented into depressive moods. Figure 6 This is a diagram illustrating a method for calculating the noise index based on the correlation between sound quality factors.

[0061] refer to Figure 2 The microphone 10 can receive the user's voice and output it as an electrical signal, and the controller 20 can receive the user's voice signal (1000) output from the microphone 10 as an electrical signal.

[0062] The controller 20 can process the user's voice and extract various sound quality factors (1100) in response to processing the user's voice.

[0063] Specifically, the processor 21 can use a sound quality factor analysis program stored in the memory 22 to extract the sound quality factors of the user's voice received from the microphone 10.

[0064] refer to Figure 3 Various sound quality factors may include at least one of sound pressure level, loudness, sharpness, roughness, and wave intensity.

[0065] It can quantify and extract each of the various sound quality factors, and assign different weights to each sound quality factor based on the correlation analysis between the sound quality factors.

[0066] Subsequently, the controller 20 can calculate the user's depression index (1200) based on at least one of a variety of sound quality factors. Specifically, as provided herein, the "depression index" refers to a measurement based on emotional factors that provide indications of the user's emotional state, such as whether the user exhibits depressive symptoms, i.e., persistent sadness and loss of interest.

[0067] For example, controller 20 can calculate a user's depression index based on at least one of sound pressure level and loudness, that is, controller 20 can calculate a user's depression index based on the change pattern of sound pressure level or loudness.

[0068] The memory 22 can store algorithms including equations about sound pressure level or loudness, and the processor 21 can calculate a depression index based on the algorithms stored in the memory 22.

[0069] As an example, memory 22 may store the following [Equations 1] to [Equations 3].

[0070] [Equation 1]

[0071] N det =N PTL -N t (1)

[0072] [Equation 2]

[0073] N obj =2.9*N' PTL +1.33 (2)

[0074] [Equation 3]

[0075] N t =1.77*n 75 *(N PTL )+0.0022 (3)

[0076] At this time, N det The measurement refers to the quantitative depression index, N. PTL N refers to the perceived instantaneous loudness (the loudness of a perceived instantaneous sound) value with corrected loudness. t The detection threshold, i.e., the detection threshold, and N obj This refers to objective evaluation, specifically the qualitative depression index, N' PTL This refers to the temporal history of perceived instantaneous loudness (the loudness of a perceived instantaneous sound), and n 75 It is a percentage value and can refer to a value used to analyze sound characteristics. n was confirmed through Jury testing. 75 With N t and N PTL The correlation.

[0077] In addition, memory 22 can store information used to determine N. PTL [Algorithm 1] for the value.

[0078] [Algorithm 1]

[0079] If N inst (i)≥N inst (i-1)

[0080] Then N PTL (i)=(1-α a )*N PTL (i-1)+α a *N inst (i)

[0081] Otherwise N PTL (i)=(1-α r )*N PTL (i-1)+α r *N inst (i) (4)

[0082] In this case, α a It can refer to the attack time constant (forward masking), and α r It can refer to the release time constant (backward masking), and α a and α r It is a value that reflects the masking effect of ambient noise in order to identify the user's voice.

[0083] The processor 21 can calculate a detection metric (i.e., a quantified depression index), identifying the user's emotional state as pleasant when the detection metric is below 0, identifying the user's emotional state as unstable when the detection metric is greater than 0 and less than or equal to 0.2, and identifying the user's emotional state as depressed when the detection metric is greater than 0.2.

[0084] In addition, the processor 21 can calculate an objective rating (i.e., a qualitative depression index) and identify emotional states that are close to a depressive state as having a higher objective rating and emotional states that are close to a pleasant state as having a lower objective rating.

[0085] That is, the depression index can include a quantitative depression index and a qualitative depression index, and when the depression index is greater than or equal to a preset value, the processor 21 can identify the user's emotional state as a depressed state.

[0086] Specifically, when the quantitative depression index is greater than or equal to the first preset value or the qualitative depression index is greater than or equal to the second preset value (in 1300, yes), the processor 21 can identify the user's emotional state as a depressed state (1400).

[0087] Conversely, when the quantitative depression index is less than a first preset value or the qualitative depression index is less than a second preset value (in 1300, no), the processor 21 can identify the user's emotional state as either unstable or pleasant. (1350). For example, when the quantitative depression index is less than 0, the processor 21 can identify the user's emotional state as pleasant, and when the quantitative depression index is greater than 0 and less than 0.2, it can identify the user's emotional state as unstable.

[0088] refer to Figure 4 The processor 21 can examine the results of calculating detection metrics and objective evaluation values ​​over time. As described above, the processor 21 can identify the user's emotional state by calculating the user's speech detection metrics and objective evaluation values ​​in real time.

[0089] although Figure 2 However, memory 22 may store as training data an algorithm related to a neural network that learns by using multiple sound quality factors and positive alpha asymmetry (FAA) values ​​(i.e., an index indicating the user’s mood), and processor 21 may use the pre-trained neural network stored in memory 22 to identify the user’s emotional state from multiple sound quality factors.

[0090] Because neural networks refer to machine learning based on the shape of neural structures capable of performing deep learning, the weights and biases corresponding to the configuration of the neural network are continuously changed to improve the reliability of learning.

[0091] As described above, controller 20 can use a pre-trained neural network to identify whether the user's emotional state is depressive.

[0092] In this context, the Russell emotion model can be used to express the user's emotional state. The Russell emotion model is represented as a two-dimensional graph based on the x-axis (positivity) and y-axis (excitement), dividing emotions into eight regions: happiness (0 degrees), excitement (45 degrees), arousal (90 degrees), pain (135 degrees), discomfort (180 degrees), depression (225 degrees), drowsiness (270 degrees), and relaxation (315 degrees). Furthermore, these eight regions are further divided into a total of 28 emotions and categorized into similar emotions belonging to the eight regions.

[0093] At this point, when using an emotion model to identify a user's emotional state, if the user's positivity and excitement at the first time point are the same as those at the second time point, then the user's emotional state at the first time point and the second time point can be identified as the same emotional state.

[0094] Refer again Figure 2The controller 20 can determine the depressive state as a first state or a second state (1500) based on the correlation between at least two of a variety of sound quality factors.

[0095] That is, even if the user's emotional state is identified as "depressed state", the controller 20 further subdivides the "depressed state" and distinguishes whether the user's depressed state is a state in which the user may feel "uncomfortable" (hereinafter referred to as the first state) or a state in which the user may feel annoyed (hereinafter referred to as the second state).

[0096] refer to Figure 5 When analyzing a user's voice in the mode shown below, it can be determined that the user's emotional state is in the first state; and when analyzing a user's voice in the mode shown above, it can be determined that the user's emotional state is in the second state.

[0097] That is, when the analysis of a user's speech shows a single peak in the frequency domain and / or time band, the user's emotional state can be judged as the first state, and if there are broadband features in the frequency domain due to the analysis of the user's speech, the user's emotional state can be judged as the second state.

[0098] According to the emotion adjustment system 1 of the implementation method, the depression state is judged as a first state and a second state by using the correlation between various sound quality factors.

[0099] Specifically, the controller 20 can calculate the noise index of the user's voice based on the correlation between sharpness, roughness, and fluctuation intensity among various sound quality factors, and determine the depressive state as a first state or a second state based on the noise index.

[0100] In this context, the noise index can include a loudness index that quantifies the inclusion of loud sounds and a loudness index that quantifies the inclusion of glissando sounds.

[0101] To this end, memory 22 can store lookup tables and / or algorithms for calculating noise index based on the correlation between various sound quality factors.

[0102] refer to Figure 6 Based on the correlation between sharpness and roughness, the correlation between roughness and fluctuation intensity, and the correlation between sharpness and fluctuation intensity, it is possible to check and derive graphs of whether the user's voice is included in the loudness region or glissando region, as well as equations for calculating the loudness index and glissando index.

[0103] The controller 20 can determine whether the user's voice is included in the loud or glissando region based on the correlation between sharpness, roughness, and fluctuation intensity. When the user's voice belongs to the loud region, the emotional state of the user identified as depressed is determined to be the first state, and when the user's voice belongs to the glissando region, the emotional state of the user identified as depressed is determined to be the second state.

[0104] In addition, controller 20 can be based on Figure 6 The equation shown calculates the loudness index and glissando index of the user's speech, and determines the depressive state as either a first state or a second state based on the ratio of the loudness index to the glissando index.

[0105] At this time, when the loudness index is greater than the glide index, the controller 20 can determine the depressed state as the first state, and when the glide index is greater than the loudness index, the controller 20 can determine the depressed state as the second state.

[0106] For example, when the roughness value is 0.269 (Asper), the undulation intensity value is 0.396 (Vacil), and the sharpness value is 1.3 (Acum), the controller 20 can use an algorithm that includes lookup tables and / or equations stored in the memory 22 to calculate a glissando index of 1.741 and a loudness index of 0.510.

[0107] As shown above, when the glissando index is calculated to be 1.741 and the loudness index is 0.510, the controller 20 is able to determine the emotional state of the user identified as being in a depressed state as the second state.

[0108] That is, ultimately, the controller 20 can identify the emotional state of a user identified as being in a depressive state as either a first state or a second state.

[0109] The controller 20 can send a control command (1600) corresponding to the user's emotional state in response to the final identification of the user's emotional state.

[0110] Therefore, the memory 22 can store the operation information of the feedback device 30 corresponding to the user's emotional state.

[0111] For example, the memory 22 can store a first control command for controlling the vibrating element 31 to vibrate at a first frequency and a first intensity as a control command corresponding to a first state, and store a second control command for controlling the vibrating element 31 to vibrate at a second frequency and a second intensity as a control command corresponding to a second state. In this case, the first frequency may be less than the second frequency, and the first intensity may be less than the second intensity.

[0112] Accordingly, when the user's emotional state is identified as a first state, the controller 20 sends a first control command to cause the vibrating element 31 to vibrate at a first frequency and a first intensity. When the user's emotional state is identified as a second state, the controller 20 can send a second control command to cause the vibrating element 31 to vibrate at a second frequency and a second intensity.

[0113] The feedback device 30 can perform operations corresponding to control commands sent from the controller 20. For example, the vibration element 31 can vibrate at a first frequency and a first intensity in response to receiving a first control command sent from the controller 20, and the vibration element 31 can vibrate at a second frequency and a second intensity in response to receiving a second control command sent from the controller 20.

[0114] According to the emotion adjustment system 1 of the embodiment, when the user's emotional state is identified as a first state, the user can be stabilized by providing the user with relatively low frequency and weak intensity vibration, and when the user's emotional state is identified as a second state, the user's motivation and / or excitement can be improved by providing the user with relatively high frequency and high intensity vibration.

[0115] Furthermore, the memory 22 can store a first control command for controlling the speaker 32 to play a first sound source as a control command corresponding to a first state, and can store a second control command for controlling the speaker 32 to play a second sound source as a control command corresponding to a second state. In this case, the bandwidth of the first sound source can be narrower than the bandwidth of the second sound source.

[0116] Accordingly, when the user's emotional state is identified as a first state, the controller 20 can send a first control command to cause the speaker 32 to play a first sound source. When the user's emotional state is identified as a second state, the controller 20 can send a second control command to cause the speaker 32 to play a second sound source.

[0117] Feedback device 30 can perform operations corresponding to control commands sent from controller 20. For example, speaker 32 can output a first sound source in response to receiving a first control command sent from controller 20, and can output a second sound source in response to receiving a second control command sent from controller 20.

[0118] According to the emotion adjustment system 1 of the embodiment, when the user's emotional state is identified as a first state, the user's stability can be promoted by providing the user with a quiet and emotionally soothing voice, and when the user's emotional state is identified as a second state, the user's level of positivity and / or excitement can be improved by providing the user with a light and moving soothing voice.

[0119] According to this disclosure, even if a user's emotional state at a first time point and their emotional state at a second time point are classified as the same emotional state (e.g., a depressed state), classifying the same emotional state into a first state and a second state through the correlation between various sound quality factors allows for a more accurate identification of the user's emotions. Consequently, more satisfactory feedback can be provided to the user.

[0120] According to this disclosure, by recognizing emotions using only the user's voice, it is possible to prevent increased costs due to the addition of components for emotion judgment.

[0121] Furthermore, users can be given more appropriate feedback by segmenting and categorizing their negative emotions.

[0122] Furthermore, the disclosed embodiments can be implemented in the form of a recording medium that stores instructions executed by a computer. Instructions in the form of program code can be stored, and when executed by a processor, the instructions can generate program modules that perform the operations of the disclosed embodiments. The recording medium can be implemented as a computer-readable recording medium.

[0123] Computer-readable recording media can include all sorts of recording media that store commands that can be interpreted by a computer. For example, computer-readable recording media can be ROM, RAM, magnetic tape, magnetic disk, flash memory, optical data storage devices, etc.

[0124] Therefore, embodiments of the present disclosure have been described to date with reference to the accompanying drawings. It will be apparent to those skilled in the art that the present disclosure can be implemented in other forms besides the exemplary embodiments described above without altering the technical concept or essential features of the present disclosure. The disclosed embodiments are illustrative and should not be considered limiting.

Claims

1. An emotion regulation system, comprising: The microphone is configured to receive the user's voice. The controller is configured to extract multiple sound quality factors in response to processing the user's voice, calculate the user's depression index based on at least one of the multiple sound quality factors, identify the user's emotional state as a depressed state when the depression index is above a preset value, determine the depressed state as a first state or a second state based on the correlation between at least two of the multiple sound quality factors, and send a control command corresponding to the emotional state of the user identified as the first state or the second state. as well as The feedback device is configured to perform the operation corresponding to the control command. The various sound quality factors include at least one of sound pressure level, loudness, sharpness, roughness, and wave intensity. The controller is configured to calculate the noise index of the user's speech based on the correlation between the sharpness, the roughness, and the fluctuation intensity, and to determine the depressive state as either the first state or the second state based on the noise index. The noise index includes a loudness index and a glide index, and The controller is configured to determine the depressive state as either the first state or the second state based on the ratio of the loudness index to the glide index.

2. The emotion adjustment system according to claim 1, wherein, The controller is configured to calculate the user's depression index based on at least one of the sound pressure level and the loudness.

3. The emotion adjustment system according to claim 2, wherein, The controller is configured to calculate the user's depression index based on the change pattern of the sound pressure level or the loudness.

4. The emotion adjustment system according to claim 1, wherein, When the loudness index is greater than the glide index, the controller is configured to determine the depressed state as the first state, and when the glide index is greater than the loudness index, the controller is configured to determine the depressed state as the second state.

5. The emotion adjustment system according to claim 4, wherein, The feedback device includes a vibration element, and The controller is configured to: when the user's emotional state is identified as the first state, send a first control command to cause the vibrating element to vibrate at a first frequency and a first intensity; and when the user's emotional state is identified as the second state, send a second control command to cause the vibrating element to vibrate at a second frequency and a second intensity; and The first frequency is less than the second frequency and the first intensity is less than the second intensity.

6. The emotion adjustment system according to claim 4, wherein, The feedback device includes a speaker, and The controller is configured to: when the user's emotional state is identified as the first state, send a first control command to cause the speaker to play a first sound source; and when the user's emotional state is identified as the second state, send a second control command to cause the speaker to play a second sound source; and The frequency band of the first sound source is narrower than that of the second sound source.

7. The emotion adjustment system according to claim 1, wherein, The controller is configured to: when the depression index is less than the preset value, identify the user's emotional state as an unstable state or a pleasant state, and send a control command corresponding to the user's emotional state that is identified as the unstable state or the pleasant state.

8. An emotion regulation method, comprising: The microphone receives the user's voice. The controller extracts various sound quality factors in response to processing the user's voice; The controller calculates the user's depression index based on at least one of the multiple sound quality factors. When the depression index is above a preset value, the controller identifies the user's emotional state as a depressed state. The controller determines the depressive state as a first state or a second state based on the correlation between at least two of the multiple sound quality factors. The controller sends a control command corresponding to the emotional state of the user identified as the first state or the second state; and The feedback device executes the operation corresponding to the control command. The various sound quality factors include at least one of sound pressure level, loudness, sharpness, roughness, and wave intensity. Specifically, classifying the depressive state as either the first state or the second state includes: The noise index of the user's speech is calculated based on the correlation between the sharpness, the roughness, and the fluctuation intensity; and Based on the noise index, the depressive state is determined to be either the first state or the second state. The noise index includes a loudness index and a glide index, and Specifically, determining the depressive state as the first state or the second state based on the noise index includes: The depressive state is determined to be either the first state or the second state based on the ratio of the loudness index to the glide index.

9. The emotion adjustment method according to claim 8, wherein, Calculating the user's depression index includes: The user's depression index is calculated based on at least one of the sound pressure level and the loudness.

10. The emotion adjustment method according to claim 9, wherein, Calculating the user's depression index includes: The user's depression index is calculated based on the change pattern of the sound pressure level or the loudness.

11. The emotion adjustment method according to claim 8, wherein, Determining the depressive state as either the first state or the second state based on the ratio of the loudness index to the glide index includes: When the loudness index is greater than the glissando index, the depressed state is determined to be the first state, and when the glissando index is greater than the loudness index, the depressed state is determined to be the second state.

12. The emotion adjustment method according to claim 11, wherein, The feedback device includes a vibration element, and The process of sending control commands corresponding to the emotional state of the user identified as being in the first state or the second state includes: When the user's emotional state is identified as the first state, a first control command is sent to cause the vibrating element to vibrate at a first frequency and a first intensity; and When the user's emotional state is identified as the second state, a second control command is sent to cause the vibrating element to vibrate at a second frequency and a second intensity; and The first frequency is less than the second frequency and the first intensity is less than the second intensity.

13. The emotion adjustment method according to claim 11, wherein, The feedback device includes a speaker, and wherein sending the control command corresponding to the emotional state of the user identified as the first state or the second state includes: When the user's emotional state is identified as the first state, a first control command is sent to cause the speaker to play a first sound source; and When the user's emotional state is identified as the second state, a second control command is sent to cause the speaker to play a second sound source; and The frequency band of the first sound source is narrower than that of the second sound source.

14. The emotion adjustment method according to claim 8, further comprising: When the depression index is less than the preset value, the user's emotional state is identified as either an unstable state or a pleasant state. and Send a control command corresponding to the emotional state of the user who is identified as either the unstable state or the pleasant state.

Images