User emotion recognition method and device, server and computer readable storage medium
Through the analysis of audio and image data, the emotional value is calculated and the audio is played and adjusted, the safety problems caused by emotional instability of drivers are solved, accurate identification and timely intervention of drivers' emotions are achieved, and driving safety is improved.
Patent Information
- Application Number
- CN202411180465.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-27
- Publication Date
- 2024-12-20
AI Technical Summary
The prior art is difficult to accurately judge the emotional stability of the driver, which leads to possible unexpected situations during driving, and a method is required to intervene in time to ensure driving safety.
By obtaining the audio and image data of the driver, calculating its audio and expression parameters, and determining the emotional value based on the weight value. If the emotions are unstable, the audio will be played and adjusted to intervene, real-time monitoring and intervention of emotions can be achieved.
Accurate identification and timely intervention of drivers' emotions is achieved, driving safety is improved, and accidents are avoided due to emotional instability.
Smart Images
Figure CN119160195A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of vehicle control technology, and in particular, to a user emotion recognition method, device, server and computer-readable storage medium. Background Art
[0002] With the development of the times, users' demand for vehicles is increasing day by day, and the emotional stability of drivers is particularly important. Facing congestion, rushing for time, cutting in to overtake, slow-moving cars, etc., if the user's emotions are unstable, accidents may occur. Therefore, how to accurately judge the driver's emotions and intervene as soon as possible when his emotions are unstable, so as to ensure driving safety, has become a problem that needs to be solved urgently. Summary of the invention
[0003] In view of this, the purpose of the present application is to provide a user emotion recognition method, device, server and computer-readable storage medium to accurately identify the driver's emotions.
[0004] In order to achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:
[0005] In a first aspect, the present invention provides a method for identifying user emotions, which is applied to a server, and the method comprises:
[0006] Acquire current user audio and current user image; the current user image includes the user's facial information;
[0007] Determine current audio parameters and current expression parameters according to the current user audio and the current user image, and calculate the current emotion value of the user according to the current audio parameters and the current expression parameters;
[0008] The current emotion value represents the current emotional stability of the user.
[0009] In an optional implementation manner, the server stores a first correspondence relationship between audio parameters and audio weight values, and a second correspondence relationship between expression parameters and expression weight values;
[0010] The determining of current audio parameters and current expression parameters according to the current user audio and the current user image, and calculating the current emotion value of the user according to the current audio parameters and the current expression parameters, includes:
[0011] Determining the current audio parameter according to the current user audio, and determining a target audio weight value corresponding to the current user audio according to the current audio parameter and the first corresponding relationship;
[0012] Determining the current expression parameter according to the current user image, and determining the target expression weight value corresponding to the current user image according to the current expression parameter and the second corresponding relationship;
[0013] The current emotion value of the user is determined according to the target audio weight value and the target expression weight value.
[0014] In an optional implementation, the audio parameter includes a volume parameter, a content type, and a content keyword, and the target audio weight value includes a target volume weight value and a target content weight value;
[0015] The determining the current audio parameter according to the current user audio, and determining the target audio weight value corresponding to the current user audio according to the current audio parameter and the first corresponding relationship, includes:
[0016] Performing volume analysis and content analysis on the current user audio to obtain a current volume parameter, a current content type, and a current content keyword of the current user audio;
[0017] Determine, according to the current volume parameter, the current content type, the current content keyword and the first corresponding relationship, a target volume weight value corresponding to the current volume parameter, a target type weight value corresponding to the current content type and a target keyword weight value corresponding to the current content keyword;
[0018] The target content weight value is determined according to the target type weight value and the target keyword weight value.
[0019] In an optional implementation, the expression parameter includes an expression emotion, and determining the current expression parameter according to the current user image includes:
[0020] Perform facial feature recognition on the current user image to obtain the user's facial features, and determine the user's current facial expression based on the facial features.
[0021] In an optional implementation, the server is in communication with the driving recorder, and at least one adjusted audio is stored in the server. The method includes:
[0022] If the current emotion value exceeds a preset threshold, the audio to be played is determined from at least one of the adjusted audios according to the current emotion value, and the driving recorder is controlled to play the audio to be played to adjust the user's emotion.
[0023] In an optional implementation manner, the server is communicatively connected to the terminal device, and the method further includes:
[0024] Obtaining the audio parameters, expression parameters and emotion values of the user within the preset time period at intervals of the preset time period;
[0025] The emotion change of the user within the preset time period is determined according to the audio parameter, the expression parameter and the emotion value, and the emotion change is sent to the terminal device.
[0026] In a second aspect, the present invention provides a user emotion recognition device, applied to a server, the device comprising:
[0027] An acquisition module, used to acquire the current user audio and the current user image; the current user image includes the user's facial information;
[0028] A determination module is used to determine current audio parameters and current expression parameters based on the current user audio and the current user image, and calculate the user's current emotion value based on the current audio parameters and the current expression parameters; the current emotion value represents the current emotional stability of the user.
[0029] In an optional implementation manner, the server stores a first correspondence relationship between audio parameters and audio weight values, and a second correspondence relationship between expression parameters and expression weight values;
[0030] The determination module is also used to determine the current audio parameters based on the current user audio, and determine the target audio weight value corresponding to the current user audio based on the current audio parameters and the first corresponding relationship; determine the current expression parameters based on the current user image, and determine the target expression weight value corresponding to the current user image based on the current expression parameters and the second corresponding relationship; determine the current emotion value of the user based on the target audio weight value and the target expression weight value.
[0031] In a third aspect, the present invention provides a server, comprising a processor and a memory, wherein the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the method described in any one of the aforementioned implementations.
[0032] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in any one of the aforementioned embodiments.
[0033] The user emotion recognition method, device, server and computer-readable storage medium provided in the embodiments of the present application can obtain the current user audio and the current user image, determine the current audio parameters and the current expression parameters according to the current user audio and the current user image, and on this basis, can comprehensively calculate the current emotion value that can characterize the current emotional stability of the user according to the current audio parameters and the current expression parameters. Therefore, the user's emotions can be accurately judged by the current emotion value, so that the user can be intervened in time when the user's emotions are unstable to ensure driving safety.
[0034] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0036] Figure 1 A block diagram of a user emotion control system is shown;
[0037] Figure 2 A block diagram of a server provided in an embodiment of the present application is shown;
[0038] Figure 3 A schematic diagram of a process of a user emotion recognition method provided by an embodiment of the present application is shown;
[0039] Figure 4 A functional module diagram of a user emotion recognition method provided in an embodiment of the present application is shown.
[0040] Icons: 10 - server; 100 - memory; 110 - processor; 120 - communication module; 20 - driving recorder; 30 - terminal device; 200 - acquisition module; 210 - determination module. DETAILED DESCRIPTION
[0041] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.
[0042] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.
[0043] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0044] Figure 1 For a block diagram of the user emotion control system, see Figure 1 The user emotion control system includes a server 10, a driving recorder 20 and a terminal device 30, and the server 10 is communicatively connected to the driving recorder 20 and the terminal device 30 respectively.
[0045] Optionally, the driving recorder 20 can be used to collect audio and images of the user during driving, and send the audio and images to the server 10, so that the server 10 can analyze the user's emotional stability based on the audio and images.
[0046] Optionally, the server 10 may send the emotion analysis of the user to the terminal device, so that the user can check his or her emotional state through the terminal device, thereby understanding his or her emotional changes and possible emotional problems during driving.
[0047] Optionally, in Figure 1 On the basis of Figure 2 For a block diagram of the server 10 provided in the embodiment of the present application, see Figure 2 The server includes a memory 100, a processor 110 and a communication module 120. The memory 100, the processor 110 and the communication module 120 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.
[0048] The memory 100 is used to store computer programs or data that can be executed by the processor. The memory 100 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc.
[0049] The processor 110 is used to read / write data or computer programs stored in the memory, and execute the computer program to implement the user emotion recognition method provided in the embodiment of the present application.
[0050] The communication module 120 is used to establish a communication connection between the server and other communication terminals through the network, and to send and receive data through the network.
[0051] It should be understood that Figure 2 The structure shown is only a schematic diagram of the server structure. The server may also include Figure 2 More or fewer components as shown, or with Figure 1 Different configurations shown. Figure 2 Each component shown in the figure can be implemented by hardware, software or a combination thereof.
[0052] Next, the above Figure 1 The server in the embodiment is the execution entity, and the user emotion recognition method provided in the embodiment of the present application is exemplarily introduced in combination with the flow chart.
[0053] Specifically, Figure 3 A flow chart of a method for identifying user emotions provided in an embodiment of the present application is shown in FIG. Figure 3 , the method comprising:
[0054] Step S20, obtaining the current user audio and the current user image.
[0055] The current user image includes the user's facial information.
[0056] In one possible implementation, the current user image may be a facial image of the user.
[0057] Optionally, the current user audio and the current user image may be collected by a driving recorder and sent to a server.
[0058] Optionally, the driving recorder may collect user audio and user image at preset time intervals.
[0059] In a possible implementation, the preset time interval may be 10 seconds.
[0060] Optionally, since a user's facial expressions generally change and sounds are emitted when the user is emotionally unstable, the driving recorder may collect the user's audio and image at the same time.
[0061] Step S21, determining current audio parameters and current expression parameters according to the current user audio and the current user image, and calculating the current emotion value of the user according to the current audio parameters and the current expression parameters.
[0062] Among them, the current emotion value represents the user's current emotional stability.
[0063] Optionally, the server may perform a comprehensive analysis on the current user audio and the current user image to obtain the current emotion value of the user.
[0064] In one possible implementation, if the current emotion value is higher, the user's emotion stability is lower and the emotion is more unstable. Conversely, if the current emotion value is lower, the user's emotion stability is higher and the emotion is more stable.
[0065] The user emotion recognition method provided in the embodiment of the present application is that the server can obtain the current user audio and the current user image, and determine the current audio parameters and the current expression parameters according to the current user audio and the current user image. On this basis, the current emotion value that can characterize the current emotional stability of the user can be comprehensively calculated according to the current audio parameters and the current expression parameters. Therefore, the user's emotions can be accurately judged through the current emotion value, so that the user can be intervened in time when the user's emotions are unstable to ensure driving safety.
[0066] Optionally, in order to timely soothe the user's emotions, at least one adjustment audio may be stored in the server, and the server may adjust the user's emotions by outputting appropriate adjustment audio when determining that the user's emotions are unstable.
[0067] Specifically, when the current emotion value exceeds a preset threshold, the server can determine the audio to be played from at least one adjusted audio according to the current emotion value, and control the driving recorder to play the audio to be played to adjust the user's emotions.
[0068] Optionally, the adjusted audio may correspond to an emotion value, and the user may set the correspondence through a driving recorder.
[0069] In a possible implementation, if the emotion value is a numerical value, the emotion value exceeding a preset threshold may be divided into a plurality of emotion value ranges, and each emotion value range may correspond to at least one adjusted audio.
[0070] In another possible implementation, if the emotion value is a weight level, such as high, medium, and low, each weight level may correspond to at least one adjusted audio.
[0071] It can be understood that if the emotion value is the weight degree, "low" represents that the user's emotion is relatively stable, "medium" represents that the emotion is relatively unstable, and "high" represents that the emotion is very unstable, so there is no need to set the corresponding adjustment audio for "low".
[0072] In this embodiment, if the emotion value is the weight degree, the preset threshold value can be "medium", that is, the user emotion is adjusted when the emotion value is high, or the preset threshold value is set to "low", that is, the user emotion is adjusted when the emotion value is medium and high. The specific setting can be made by the user according to the actual application situation, and this application does not make too many restrictions on this.
[0073] Optionally, if there are multiple adjustable audios corresponding to the current emotion value, for example, the current emotion value is "high", and "high" corresponds to multiple adjustable audios, then the audio selected by the user in advance can be determined as the audio to be played, or the default audio can be set as the audio to be played, or an audio can be randomly selected as the audio to be played.
[0074] In this embodiment, the adjusted audio may be music generated in advance by the server according to the emotional music model, or may be audio uploaded by the user.
[0075] Optionally, the server may also store a first correspondence between audio parameters and audio weight values, and a second correspondence between expression parameters and expression weight values.
[0076] Optionally, the first corresponding relationship and the second corresponding relationship can be set by the user through the driving recorder according to the actual situation of the user.
[0077] For example, if user A is emotionally unstable and his voice becomes louder, the weight value corresponding to the volume can be set to a larger value; user B is emotionally unstable when he feels sad, so the weight value corresponding to the sad expression can be set to a larger value.
[0078] It can be understood that by the user setting the first corresponding relationship and the second corresponding relationship by himself, the weight value can be more adapted to the actual situation when the user is emotionally unstable, thereby ensuring accurate identification of whether the user has emotional instability problems.
[0079] Based on this, a possible implementation method is provided below on how to determine the current audio parameters and the current expression parameters according to the current user audio and the current user image, and calculate the current emotion value of the user according to the current audio parameters and the current expression parameters.
[0080] Specifically, the server can determine the current audio parameters based on the current user audio, and determine the target audio weight value corresponding to the current user audio based on the current audio parameters and the first correspondence, and determine the current expression parameters based on the current user image, and determine the target expression weight value corresponding to the current user image based on the current expression parameters and the second correspondence, so that the user's current emotion value can be determined based on the target audio weight value and the target expression weight value.
[0081] In one example, if the weight value is represented by a degree of instability, such as "low", "medium", and "high", the server can determine that the user's current emotional value is "high" when any of the target audio weight value and the target expression weight value is represented as "high"; when the target audio weight value and the target expression weight value are both "low", the user's current emotional value is determined to be "low"; when the target audio weight value and the target expression weight value are both "medium", the user's current emotional value is determined to be "medium"; when the target audio weight value and the target expression weight value include both "medium" and "low", the user's current emotional value is determined to be "medium" or "low" according to the specific circumstances.
[0082] In another example, if the weight value is a specific numerical value, the sum of the target audio weight value and the target expression weight value may be determined as the current emotion value of the user.
[0083] Next, we will introduce how to determine the target audio weight value and the target expression weight value for the current user audio and the current user image respectively.
[0084] Optionally, for audio, both the audio content and the audio volume can reflect a person's emotions. For example, a user may cry or yell loudly when sad, making crying sounds such as "woohoo", cheer or scream loudly when excited, making exclamations such as "ah!" and "wow!", and make rapid breathing sounds or swearing when angry. Therefore, the server can analyze the current user audio from two aspects: content and volume.
[0085] Optionally, the audio content may include content type and content keywords, wherein the content type may be the emotion type of the audio, for example, including "normal", "sad", "excited", "angry", etc., and the content keywords may include interjections such as "wow", "ah", "go away", "sad", etc., and words that can represent emotions.
[0086] It can be understood that, based on this, the audio parameters may include volume parameters, content types and content keywords, and the target audio weight value may include a target volume weight value and a target content weight value.
[0087] On this basis, a possible implementation method is given below for how to determine the current audio parameters according to the current user audio, and determine the target audio weight value corresponding to the current user audio according to the current audio parameters and the first corresponding relationship.
[0088] Optionally, the server can perform volume analysis and content analysis on the current user audio to obtain the current volume parameter, current content type and current content keyword of the current user audio, and determine the target volume weight value corresponding to the current volume parameter, the target type weight value corresponding to the current content type and the target keyword weight value corresponding to the current content keyword based on the current volume parameter, the current content type, the current content keyword and the first correspondence.
[0089] Optionally, the current volume parameter may be a decibel value of the current user audio.
[0090] In one example, the server can perform content analysis on the current user audio. If it is analyzed that the audio contains rapid breathing sounds accompanied by roars, etc., then it can be determined that the current content type of the current user audio is "angry", and the server can also convert the current user audio into voice text, and perform keyword matching based on the voice text, so as to obtain the content keywords corresponding to the current user audio.
[0091] It is understandable that in order to prevent errors in audio emotion recognition, the audio volume, content type and content keywords can be combined to perform comprehensive recognition of the audio, thereby improving the accuracy of emotion recognition.
[0092] In this embodiment, the first corresponding relationship may include a corresponding relationship between a decibel value and a volume weight value, a corresponding relationship between a content type and a type weight value, and a corresponding relationship between a content keyword and a keyword weight value.
[0093] In this embodiment, the server can determine the target volume weight value corresponding to the current volume parameter based on the current volume parameter and the correspondence between the decibel value and the volume weight value; determine the target type weight value corresponding to the current content type based on the current content type and the correspondence between the content type and the type weight value; determine the target keyword weight value corresponding to the current content keyword based on the current content keyword and the correspondence between the content keyword and the keyword weight value.
[0094] In this embodiment, since both the content type and the content keyword are related to the audio content, the server also needs to comprehensively determine the target content weight value based on the target type weight value and the target keyword weight value.
[0095] Optionally, the server may use the target type weight value and the target keyword weight value, whichever has a higher weight value, as the target content weight value.
[0096] For example, if the target type weight value is "medium", but the target keyword weight value is "high", the server may determine that the target content weight value is "high".
[0097] Optionally, for an image, since the user's expression can reflect the user's emotions, the server can determine a corresponding weight value according to the emotion represented by the user's expression.
[0098] Based on this, the expression parameter may be the user's expression emotion.
[0099] In this embodiment, the server may perform facial feature recognition on the current user image to obtain the user's facial features, and determine the user's current facial expression based on the facial features.
[0100] Optionally, the facial feature may be a facial feature of the user.
[0101] In one example, if the user is in a sad mood, the eyebrows may bend inward, the eyes may unconsciously squint or close, the nose may be slightly wrinkled, the lips may be tightly closed or slightly tremble, the corners of the mouth may droop, the head may droop, etc.
[0102] If the user is angry, the eyes may widen, the eyebrows may be furrowed or lowered, the nose may wrinkle, the nostrils may flare, the lips may be pulled back, the mouth may be tightly closed, the facial muscles may become tense and stiff, and the head may tilt forward.
[0103] If the user is in a state of fear, the eyes may open wide, the pupils may dilate, the eyebrows may raise, the brows may furrow, the mouth may open, the lips may tremble, the facial muscles may become stiff, the cheeks may become tense, and the head may tilt back.
[0104] If the user is in a state of surprise, the eyes may suddenly open wide, the pupils may dilate, the eyebrows may rise, especially the inner eyebrows, forming a high arch, the nose may not react very obviously, the mouth may open to form an "O" shape, the lips may be slightly parted, and the head may be slightly tilted back or to one side.
[0105] If the user is in an excited mood, the eyes may open wider, the eyebrows may raise, the mouth may open to show a smile and even show teeth, the lips may be raised, the head may be raised confidently, etc.
[0106] If the user is in a state of disgust, the eyes may narrow, the eyebrows may wrinkle inward, the brows may be furrowed, the nose may wrinkle, the nostrils may flare, the mouth may close, the lips may be pursed, the facial muscles may appear tense, and the head may tilt back or to one side.
[0107] It can be understood that the server can identify the corresponding emotion of the user according to the facial features of the user and obtain the user's current expression emotion.
[0108] In this embodiment, the second corresponding relationship may include a corresponding relationship between facial expressions and facial expression weight values.
[0109] Optionally, the server may directly determine the corresponding target expression weight value according to the recognized current expression emotion and the second corresponding relationship.
[0110] Furthermore, in the present embodiment, considering that the target audio weight value includes the target volume weight value and the target content weight value, when determining the user's current emotion value based on the target audio weight value and the target expression weight value, if the target audio weight value and the target expression weight value include both "medium" and "low", the server can determine whether the user's current emotion value is "medium" or "low" based on the number of "medium" and "low".
[0111] In one possible implementation, if any two of the target audio weight value and the target expression weight value are "medium", the user's current emotion value can be determined to be "medium"; if any two of the target audio weight value and the target expression weight value are "low", the user's current emotion value can be determined to be "low".
[0112] For example, if the target volume weight value and the target content weight value are both "medium", and the target expression weight value is "low", it can be determined that the user's current emotion value is "medium".
[0113] Optionally, in order to count the user's emotional changes and facilitate user search and analysis, the server can also obtain the user's audio parameters, expression parameters and emotional values within the preset time period at preset time intervals, and determine the user's emotional changes within the preset time period based on the audio parameters, expression parameters and emotional values, and then send the emotional changes to the terminal device.
[0114] It is understandable that the preset duration can be set according to actual application requirements, such as one day or one hour.
[0115] In this embodiment, the server may obtain all audio parameters, expression parameters, and emotion values acquired within the preset time period, and analyze the emotion changes of the user.
[0116] Optionally, the emotion change situation can be represented in the form of a table, a graph, etc.
[0117] In one example, the emotion change situation may include the user's first emotion change situation determined according to the volume within a preset time length, the user's second emotion change situation determined according to the audio content within a preset time length, the user's third emotion change situation determined according to the facial expression within a preset time length, and the user's fourth emotion change situation determined according to the emotion value within a preset time length.
[0118] In this example, the first emotion change situation, the second emotion change situation, and the third emotion change situation may all include specific weight values, emotional states, change times, and change times, etc.
[0119] For example, the third emotion change situation may include specific emotions, such as sadness, anger, excitement, etc., as well as the weight value, occurrence time, and occurrence frequency corresponding to the specific emotion.
[0120] It is understandable that users can check their emotional changes through terminal devices to determine which emotion appears most frequently and what the most significant feature of the emotion is, such as volume, audio content or expression, so as to make targeted adjustments to the first corresponding relationship and the second corresponding relationship, so that subsequent server recognition of user emotions can be more accurate.
[0121] In order to perform the corresponding steps in the above embodiments and various possible methods, a method for implementing a method for identifying user emotions is provided below. Figure 4 , Figure 4 A functional module diagram of a user emotion recognition method provided in an embodiment of the present application. It should be noted that the basic principle and technical effects of the user emotion recognition method provided in this embodiment are the same as those of the above-mentioned embodiments. For the sake of brief description, for parts not mentioned in this embodiment, reference can be made to the corresponding contents in the above-mentioned embodiments. The user emotion recognition method includes: an acquisition module 200 and a determination module 210.
[0122] The acquisition module 200 is used to acquire the current user audio and the current user image; the current user image includes the user's facial information.
[0123] It can be understood that the acquisition module 200 can also be used to execute the above step S20.
[0124] The determination module 210 is used to determine the current audio parameters and the current expression parameters according to the current user audio and the current user image, and calculate the current emotion value of the user according to the current audio parameters and the current expression parameters; the current emotion value represents the current emotional stability of the user.
[0125] It can be understood that the determination module 210 can also be used to execute the above step S20.
[0126] Optionally, the determination module 210 is also used to determine the current audio parameters based on the current user audio, and determine the target audio weight value corresponding to the current user audio based on the current audio parameters and the first corresponding relationship; determine the current expression parameters based on the current user image, and determine the target expression weight value corresponding to the current user image based on the current expression parameters and the second corresponding relationship; determine the user's current emotion value based on the target audio weight value and the target expression weight value.
[0127] Optionally, the determination module 210 is also used to perform volume analysis and content analysis on the current user audio to obtain the current volume parameter, current content type and current content keyword of the current user audio; determine the target volume weight value corresponding to the current volume parameter, the target type weight value corresponding to the current content type and the target keyword weight value corresponding to the current content keyword according to the current volume parameter, the current content type, the current content keyword and the first correspondence; determine the target content weight value according to the target type weight value and the target keyword weight value.
[0128] Optionally, the determination module 210 is further configured to perform facial feature recognition on the current user image, obtain the user's facial features, and determine the user's current facial expression based on the facial features.
[0129] Optionally, the determination module 210 is also used to determine the audio to be played from at least one adjusted audio according to the current emotion value if the current emotion value exceeds a preset threshold, and control the driving recorder to play the audio to be played to adjust the user's emotions.
[0130] Optionally, the determination module 210 is further configured to obtain the user's audio parameters, expression parameters and emotion values within a preset time period at intervals; determine the user's emotion changes within the preset time period according to the audio parameters, expression parameters and emotion values, and send the emotion changes to the terminal device. Optionally, the above modules can be stored in the form of software or firmware. Figure 2 The memory shown in the figure or fixed in the operating system (OS) of the server, and can be Figure 2 Meanwhile, the data and program codes required for executing the above modules can be stored in the memory.
[0131] The embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the user emotion recognition method provided by the embodiment of the present application can be implemented.
[0132] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and a part of the module, program segment or code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0133] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
[0134] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0135] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A method for identifying user emotions, characterized in that: Applied to a server, the method comprises: Acquire current user audio and current user image; the current user image includes the user's facial information; Determine current audio parameters and current expression parameters according to the current user audio and the current user image, and calculate the current emotion value of the user according to the current audio parameters and the current expression parameters; The current emotion value represents the current emotional stability of the user.
2. The method according to claim 1, characterized in that The server stores a first correspondence between audio parameters and audio weight values, and a second correspondence between expression parameters and expression weight values; The determining of current audio parameters and current expression parameters according to the current user audio and the current user image, and calculating the current emotion value of the user according to the current audio parameters and the current expression parameters, includes: Determining the current audio parameter according to the current user audio, and determining a target audio weight value corresponding to the current user audio according to the current audio parameter and the first corresponding relationship; Determining the current expression parameter according to the current user image, and determining the target expression weight value corresponding to the current user image according to the current expression parameter and the second corresponding relationship; The current emotion value of the user is determined according to the target audio weight value and the target expression weight value.
3. The method according to claim 2, characterized in that The audio parameters include volume parameters, content type and content keywords, and the target audio weight value includes a target volume weight value and a target content weight value; The determining the current audio parameter according to the current user audio, and determining the target audio weight value corresponding to the current user audio according to the current audio parameter and the first corresponding relationship, includes: Performing volume analysis and content analysis on the current user audio to obtain a current volume parameter, a current content type, and a current content keyword of the current user audio; Determine, according to the current volume parameter, the current content type, the current content keyword and the first corresponding relationship, a target volume weight value corresponding to the current volume parameter, a target type weight value corresponding to the current content type and a target keyword weight value corresponding to the current content keyword; The target content weight value is determined according to the target type weight value and the target keyword weight value.
4. The method according to claim 2, characterized in that: The expression parameter includes an expression emotion, and determining the current expression parameter according to the current user image includes: Perform facial feature recognition on the current user image to obtain the user's facial features, and determine the user's current facial expression based on the facial features.
5. The method according to claim 1, characterized in that The server is in communication with the driving recorder, and at least one adjusted audio is stored in the server. The method includes: If the current emotion value exceeds a preset threshold, the audio to be played is determined from at least one of the adjusted audios according to the current emotion value, and the driving recorder is controlled to play the audio to be played to adjust the user's emotion.
6. The method according to claim 1, characterized in that The server is in communication connection with the terminal device, and the method further comprises: Obtaining the audio parameters, expression parameters and emotion values of the user within the preset time period at intervals of the preset time period; The emotion change of the user within the preset time period is determined according to the audio parameter, the expression parameter and the emotion value, and the emotion change is sent to the terminal device.
7. A user emotion recognition device, characterized in that: Applied to a server, the device comprises: An acquisition module, used to acquire the current user audio and the current user image; the current user image includes the user's facial information; A determination module is used to determine current audio parameters and current expression parameters based on the current user audio and the current user image, and calculate the user's current emotion value based on the current audio parameters and the current expression parameters; the current emotion value represents the current emotional stability of the user.
8. The device according to claim 7, characterized in that The server stores a first correspondence between audio parameters and audio weight values, and a second correspondence between expression parameters and expression weight values; The determination module is further configured to determine the current audio parameter according to the current user audio, and determine the target audio weight value corresponding to the current user audio according to the current audio parameter and the first corresponding relationship; Determining the current expression parameter according to the current user image, and determining the target expression weight value corresponding to the current user image according to the current expression parameter and the second corresponding relationship; The current emotion value of the user is determined according to the target audio weight value and the target expression weight value.
9. A server, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the method according to any one of claims 1 to 6.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.