A model determination method and device for user emotion recognition

By using keywords to filter search content in the input method and training the emotion recognition model with input behavior data, the problem of insufficient emotional information in the user input content is solved, and more accurate emotion recognition and personalized services are achieved.

CN112347774BActive Publication Date: 2024-11-01BEIJING SOGOU TECHNOLOGY DEVELOPMENT CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201910722217.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-08-06
Publication Date
2024-11-01
Estimated Expiration
2040-04-28

AI Technical Summary

Technical Problem

In the prior art, there is less information that reflects the user's actual emotions in the content input by the user, resulting in inaccurate emotional recognition results.

Method used

Determine keywords through user input information, use search engines to obtain relevant search results, filter out search content with high correlation with keywords, analyze emotional identifiers, and form the first type of training sample, which is used to train emotions recognition models, combine input behavior and The second and third types of training samples of input information can improve the accuracy of the recognition of the model.

Benefits of technology

It improves the accuracy and efficiency of emotion recognition, can more accurately identify users' emotions, and provide personalized services.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112347774B_ABST
    Figure CN112347774B_ABST
Patent Text Reader

Abstract

The embodiment of the present application discloses a model determination method and device for user emotion recognition, which determines keywords based on input information generated by a user using an input method, obtains corresponding search results in a search engine through the keywords, and the search content in the search results can analyze the corresponding emotions, so a first type of training samples including search content that meets the conditions of keyword relevance and corresponding emotion identifiers are determined, and an emotion recognition model for a target user is trained through the first type of training samples. Since a large number of search contents are obtained through keyword search and carry rich information related to emotions, the emotion identifiers analyzed through the search contents not only guarantee accuracy but also have high efficiency, the emotion recognition model trained by a large number of first type training samples with relatively accurate emotion identifiers can accurately recognize the user's emotions based on the user's input information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a model determination method and device for user emotion recognition. Background Art

[0002] In addition to providing users with content input functions, input methods can also provide users with emotion recognition functions so that users can understand their emotions during the input process. Input methods can also provide users with corresponding personalized services based on their emotions, such as content recommendations, candidate item adjustments, etc.

[0003] At present, one emotion recognition method is to determine the user's emotion based on the content input by the user through the input method, that is, by analyzing the content input by the user, the emotion of the user when inputting the content is determined.

[0004] However, there is little information in the user input that reflects the user's actual emotions, and it is difficult to obtain accurate emotion recognition results only through the input content. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides a method and device for determining a model for user emotion recognition, and the determined model can accurately recognize the user's emotions based on the user's input information.

[0006] The embodiments of the present application disclose the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides a model determination method for user emotion recognition, the method comprising:

[0008] Determine keywords based on input information;

[0009] Determine a first type of training samples according to the search results corresponding to the keyword, wherein the first type of training samples includes search content in the search results whose relevance to the keyword meets a condition, and an emotion identifier determined by the search content;

[0010] An emotion recognition model is trained according to the first type of training samples, and the emotion recognition model is used to determine the emotion of the target user when the target input information is input according to the target input information of the target user.

[0011] Optionally, the method further includes:

[0012] Determining a second type of training samples according to the input behavior data, wherein the second type of training samples includes the input behavior and the emotion identifier corresponding to the input behavior;

[0013] The step of training the emotion recognition model according to the first type of training samples comprises:

[0014] An emotion recognition model is trained according to the first category of training samples and the second category of training samples.

[0015] Optionally, the method further includes:

[0016] Determine a third type of training sample according to the input information, wherein the third type of training sample includes the input information and an emotion identifier corresponding to the input information;

[0017] The step of training the emotion recognition model according to the first type of training samples comprises:

[0018] The emotion recognition model is trained according to the first category of training samples and the third category of training samples.

[0019] Optionally, the method further includes:

[0020] Acquire target input information of the target user;

[0021] Identify the emotion identifier corresponding to the target input information according to the emotion recognition model;

[0022] The emotion of the target user when inputting the target input information is determined according to the target input information and the emotion identifier corresponding to the target input information.

[0023] Optionally, the method further includes:

[0024] According to the input information of the target user within a predetermined period of time, the emotion trend of the target user within the predetermined period of time is determined by the emotion recognition model.

[0025] In a second aspect, an embodiment of the present application provides a model determination device for user emotion recognition, the device comprising a first determination unit, a second determination unit, and a training unit:

[0026] The first determining unit is used to determine keywords according to input information;

[0027] The second determining unit is used to determine a first type of training samples according to the search results corresponding to the keyword, wherein the first type of training samples includes search content in the search results whose relevance to the keyword meets a condition, and an emotion identifier determined by the search content;

[0028] The training unit is used to train an emotion recognition model according to the first type of training samples, and the emotion recognition model is used to determine the emotion of the target user when the target input information is input according to the target input information of the target user.

[0029] Optionally, the device further includes a third determining unit:

[0030] The third determining unit is used to determine a second type of training samples according to the input behavior data, wherein the second type of training samples includes an input behavior and an emotion identifier corresponding to the input behavior;

[0031] The training unit is further used to train an emotion recognition model according to the first type of training samples and the second type of training samples.

[0032] Optionally, the device further includes a fourth determining unit:

[0033] The fourth determining unit is used to determine a third type of training samples according to the input information, wherein the third type of training samples includes the input information and an emotion identifier corresponding to the input information;

[0034] The training unit is further used to train an emotion recognition model according to the first category of training samples and the third category of training samples.

[0035] Optionally, the device further includes an acquisition unit, a recognition unit and an emotion determination unit:

[0036] The acquisition unit is used to acquire target input information of the target user;

[0037] The recognition unit is used to recognize the emotion identifier corresponding to the target input information according to the emotion recognition model;

[0038] The emotion determination unit is used to determine the emotion of the target user when inputting the target input information according to the target input information and the emotion identifier corresponding to the target input information.

[0039] Optionally, the device further comprises a trend determination unit:

[0040] The trend determination unit is used to determine the emotional trend of the target user within the predetermined time period through the emotion recognition model according to the input information of the target user within the predetermined time period.

[0041] In a third aspect, an embodiment of the present application provides a model determination device for user emotion recognition, comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the following operations:

[0042] Determine keywords based on input information;

[0043] Determine a first type of training samples according to the search results corresponding to the keyword, wherein the first type of training samples includes search content in the search results whose relevance to the keyword meets a condition, and an emotion identifier determined by the search content;

[0044] An emotion recognition model is trained according to the first type of training samples, and the emotion recognition model is used to determine the emotion of the target user when the target input information is input according to the target input information of the target user.

[0045] Optionally, the processor is further configured to execute the one or more programs including instructions for performing the following operations:

[0046] Determining a second type of training samples according to the input behavior data, wherein the second type of training samples includes the input behavior and the emotion identifier corresponding to the input behavior;

[0047] The step of training the emotion recognition model according to the first type of training samples comprises:

[0048] An emotion recognition model is trained according to the first category of training samples and the second category of training samples.

[0049] Optionally, the processor is further configured to execute the one or more programs including instructions for performing the following operations:

[0050] Determine a third type of training sample according to the input information, wherein the third type of training sample includes the input information and an emotion identifier corresponding to the input information;

[0051] The step of training the emotion recognition model according to the first type of training samples comprises:

[0052] The emotion recognition model is trained according to the first category of training samples and the third category of training samples.

[0053] Optionally, the processor is further configured to execute the one or more programs including instructions for performing the following operations:

[0054] Acquire target input information of the target user;

[0055] Identify the emotion identifier corresponding to the target input information according to the emotion recognition model;

[0056] The emotion of the target user when inputting the target input information is determined according to the target input information and the emotion identifier corresponding to the target input information.

[0057] Optionally, the processor is further configured to execute the one or more programs including instructions for performing the following operations:

[0058] According to the input information of the target user within a predetermined period of time, the emotion trend of the target user within the predetermined period of time is determined by the emotion recognition model.

[0059] In a fourth aspect, an embodiment of the present application provides a machine-readable medium having instructions stored thereon, which, when executed by one or more processors, causes the device to execute the method described in the first aspect.

[0060] It can be seen from the above technical solution that keywords are determined based on the input information generated by the user using the input method, and the determined keywords can obtain corresponding search results through the search engine. According to the search content in the search results, the emotions corresponding to the search content can be analyzed, so the first type of training samples including search content that meets the conditions of keyword relevance and corresponding emotion identifiers can be determined, and the emotion recognition model is trained through the first type of training samples. The emotion recognition model is used to determine the emotion of the target user when the target input information is input according to the target input information of the target user. Since the number of search contents obtained through keyword search is large and carries rich information related to emotions, the emotion identifiers analyzed through the search content not only guarantee accuracy but also have high efficiency. The emotion recognition model trained by a large number of first type training samples with relatively accurate emotion identifiers can accurately identify the user's emotions according to the user's input information. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0062] Figure 1 A method flow chart of a model determination method for user emotion recognition provided in an embodiment of the present application;

[0063] Figure 2 A device structure diagram of a model determination device for user emotion recognition provided in an embodiment of the present application;

[0064] Figure 3 A block diagram of a model determination device for user emotion recognition provided in an embodiment of the present application;

[0065] Figure 4 A block diagram of a server provided in accordance with an embodiment of the present application. DETAILED DESCRIPTION

[0066] The embodiments of the present application are described below in conjunction with the accompanying drawings.

[0067] Currently, input methods mainly identify users' emotions through their input content. However, there is little information in the user input content that reflects the user's actual emotions, and it is difficult to obtain accurate emotion recognition results only through the input content.

[0068] To this end, an embodiment of the present application provides a model determination method for user emotion recognition, in which a search result corresponding to a keyword is introduced in order to determine a model that can accurately recognize user emotions, and the keyword is determined by the user's input information. Since the search results can obtain search content corresponding to the keyword and carrying richer emotional information, the corresponding emotion identifier can be analyzed more accurately based on the search content. Therefore, the first type of training samples determined by the search results are of higher quality and larger in number. Training the emotion recognition model with the first type of training samples can obtain better training results. Therefore, the emotion recognition model can accurately identify the user's emotions based on the user's input information.

[0069] The embodiments of the present application can be applied to a data processing device having data processing capabilities, and the data processing device can be, for example, a terminal device or a server. When the data processing device is a terminal device, the terminal device can specifically be a smart phone, a computer, a personal digital assistant (PDA), a tablet computer, etc. When the data processing device is a server, the server can be an independent server or a cluster server.

[0070] In some possible application scenarios, the data processing device may include a terminal device and a server. The training of the emotion recognition model may be performed by a server with stronger processing power, and the terminal device may obtain the trained emotion recognition model from the server and perform emotion recognition on the actual input of a local user (e.g., a target user) locally using the emotion recognition model.

[0071] Next, attach Figure 1 A model determination method for user emotion recognition provided in an embodiment of the present application is described, and the method includes:

[0072] S101: Determine keywords according to input information.

[0073] The input information involved in this step can be input information of a certain user, or it can be input information of no distinction between users, that is, it can be input information of multiple users. The user's input method is not limited in this application, and the above input information can be input through a keyboard, a virtual keyboard, a handwriting area, voice, etc.

[0074] The input information may include information related to the input content when the user uses the input method to input content. The input information may be input information within a certain period of history. The semantic meaning of the input information may reflect the emotional information of the user when inputting the input content related to the input information to a certain extent.

[0075] The input content may include the content on the screen determined by the user through the input method, and the input content may also include the content sent by the other end in the chat scene with other users or the content interacted in the chat scene, etc. The aforementioned content may include text, symbols, emoticons, pictures, etc. In this case, the input information may include such input content, and may further include user input, location information when obtaining such input content, context content, and body parameters of the user who inputs such content, etc.

[0076] For this type of input information, if you want to determine keywords based on the input information, you can segment the input information and use the segmented words as keywords, or you can extract names of people, event names, etc. from the segmented words as keywords. When the input information is an expression, the text explanation corresponding to the expression or the meaning expressed by the expression can be used as a keyword.

[0077] The input content may also include search terms input by the user for searching through the input method. In this case, the input information may be such input content. For such input information, each input information may be used as a keyword, or the keyword may be determined after further word segmentation.

[0078] S102: Determine a first type of training samples according to the search results corresponding to the keyword.

[0079] After the keyword is determined in S101, a search may be performed based on the keyword to obtain the corresponding search result. The search may be performed based on the keyword through any search engine or through a search function provided by the input method.

[0080] The first type of training samples can be determined based on the search results.

[0081] First, it should be noted that in the embodiments of the present application, the training samples are all used to train the emotion recognition model, and according to the different sources of the training samples, they can be distinguished by the label of "class", such as the first class of training samples, the second class of training samples, etc.

[0082] In S102, the training samples determined according to the search results corresponding to the keyword may be first-category training samples, which include search contents in the search results whose relevance to the keyword satisfies the condition, and emotion identifiers determined by the search contents.

[0083] The search results obtained by searching keywords include multiple search contents, which are all related to the keywords to a certain extent. However, since the purpose of searching keywords is to obtain more information related to the keywords, the emotional identifiers related to the user input information are determined through this information. Therefore, among a relatively large number of search contents, it is necessary to filter out search contents with high relevance to the keywords, or with high relevance to the user input information. The emotion-related information reflected in such search contents can be more consistent with the emotions of the input information related to such search contents. For example, the first few (such as the top n) search contents in the search results corresponding to the keywords can be filtered out as search contents with high relevance to the user input information.

[0084] Therefore, the emotion identifier determined by searching the content can more accurately reflect the emotion of the input information related thereto.

[0085] The embodiments of the present application do not limit the method of determining the emotion identification through the search content. For example, it can be determined by analyzing the semantics of the search content and other information, or it can be determined by manual labeling.

[0086] S103: Training an emotion recognition model according to the first type of training samples.

[0087] An emotion recognition model may be trained through the first type of training samples, and the emotion recognition model is used to determine the emotion of the target user when the target input information is input according to the target input information of the target user.

[0088] It should be noted that for keywords in the form of short text, there will be a lot of redundant information in the corresponding search results, which will affect the final model classification effect. Therefore, the dimensionality reduction algorithm can be used for the filtered search content, and at the same time, attention can be paid to some useful features in the search content during training to improve the training effect.

[0089] The target user is the user whose emotion needs to be recognized during input, and the target input information is the input information input by the target user. The target input information may be input information input by the target user once, or input information input in a period of time. The target input information may be obtained periodically, or may be obtained sequentially according to the input of the target user, or may be obtained in real time according to the input of the target user, or may be obtained randomly from the input information of the target user.

[0090] In an embodiment of the present application, the emotion recognition model may belong to a classification model, for example, it may be a long short-term memory (LSTM) model, or it may be a combination of an LSTM model and an attention model, or it may be a convolutional neural network (CNN) model, etc.

[0091] The emotion recognition model can derive the emotion corresponding to the target input information by analyzing the target user's target input information. The emotion can be expressed in the form of probability. For example, the probability that the target user is happy when entering the target input information is 80%, and the probability that the target user is surprised is 20%.

[0092] It can be seen that keywords are determined based on the input information generated by the user using the input method, and the determined keywords can obtain corresponding search results through the search engine. Based on the search content in the search results, the emotions corresponding to the search content can be analyzed. Therefore, the first type of training samples including search content that meets the conditions of keyword relevance and corresponding emotion identifiers can be determined, and the emotion recognition model is trained through the first type of training samples. The emotion recognition model is used to determine the emotion of the target user when the target input information is input according to the target input information of the target user. Since the number of search contents obtained through keyword search is large and carries rich information related to emotions, the emotion identifiers analyzed through the search content not only guarantee accuracy but also have high efficiency. The emotion recognition model trained by a large number of first-type training samples with relatively accurate emotion identifiers can accurately identify the user's emotions based on the user's input information.

[0093] In the embodiment of the present application, in addition to using the first type of training samples to train the emotion recognition model, the emotion recognition model may also be trained using training samples from more sources to improve the recognition accuracy of the emotion recognition model.

[0094] In a possible implementation, a second type of training samples may also be determined according to the input behavior data.

[0095] Input behavior data refers to data related to user input behavior when the user inputs content through the input method. For example, it may include the withdrawal of input strings, deletion of input content, input interval, sentence length in input content, pressure and duration during input, etc. It may also further include the input scenario when the user performs input behavior, etc.

[0096] Input behavior data can reflect the user's emotional information when inputting content to a certain extent. For example, if the user is impatient or unhappy during the input process, they are more likely to withdraw and delete more often; if the user is angry or surprised during the input process, they are more likely to use short sentences and press harder.

[0097] The second type of training samples determined based on the input behavior data may include the input behavior and the emotion identifier corresponding to the input behavior. The embodiment of the present application does not limit how the emotion identifier is determined based on the input behavior data, for example, it may be automatically determined based on a preset rule, or it may be determined by manual annotation.

[0098] In this implementation, S103 may be training an emotion recognition model according to the first type of training samples and the second type of training samples.

[0099] By further introducing the second type of training samples during the training process, the emotion recognition model can be trained in multiple dimensions. The second type of training samples can carry information that is closer to the user's actual behavior. This information can reflect the user's emotions from the perspective of input behavior, thereby improving the model training accuracy.

[0100] In addition to introducing training samples from the perspective of user input behavior for model training, in a possible implementation, user input information may also be introduced to determine training samples.

[0101] As described in S101, the semantic meaning of the user input information can reflect the user's emotional information when inputting the input content related to the input information to a certain extent. In other words, the user's emotions when inputting can be intuitively reflected in the input content in some cases.

[0102] Therefore, in a possible implementation, a third type of training sample can be determined based on the input information, and the third type of training sample includes the input information and the emotion identifier corresponding to the input information. The embodiment of the present application does not limit how the emotion identifier is determined by the input information, for example, it can be automatically determined by a preset rule, or it can be determined by manual annotation.

[0103] In this implementation, S103 may be training an emotion recognition model according to the first category of training samples and the third category of training samples.

[0104] By further introducing the third type of training samples during the training process, the emotion recognition model can be trained in multiple dimensions. The third type of training samples can carry emotional information that is closer to the user's actual input. This information can reflect the user's emotions from the perspective of the user's actual input, thereby improving the model training accuracy.

[0105] It should be noted that in addition to the two possible implementation methods mentioned above, in addition to introducing training samples from different sources for model training, the model training method of different training sample combinations can also be implemented in the embodiments of the present application. For example, S103 can further train the emotion recognition model based on the first type of training samples, the second type of training samples and the third type of training samples to achieve better training results.

[0106] The selection of the above training methods may be related to the specific application scenario, the processing capability of the data processing device, or the training accuracy requirements.

[0107] After introducing the model training, the specific usage process of the emotion recognition model is explained next.

[0108] When it is necessary to perform emotion recognition on the target user during the input process, it can be achieved through the following process:

[0109] S201: Acquire target input information of the target user.

[0110] S202: Identify the emotion identifier corresponding to the target input information according to the emotion recognition model.

[0111] S203: Determine the emotion of the target user when inputting the target input information according to the target input information and the emotion identifier corresponding to the target input information.

[0112] After the emotion identifier corresponding to the target input information is determined by the emotion recognition model, the emotion corresponding to the target input information, or the emotion of the target user when inputting the target input information, can be directly determined according to the emotion identifier.

[0113] The target input information can also be referred to when determining emotions. The purpose of introducing the target input information is that, since the target input information can carry related information such as the context corresponding to the input content, this information can to a certain extent reflect the actual emotions of the target user when entering the target input information. For example, for a piece of text, based on the context input by some users, it can be determined that these users were not in a good mood when entering the text, while some users were in a positive mood when entering the text.

[0114] Therefore, when the target input information itself can reflect relatively obvious emotional information, it can help improve the accuracy of determining the emotion of the target user when inputting the target input information.

[0115] In the actual process of determining emotions, corresponding weights may be set for the target input information and the emotion identifier. In a possible implementation, since the emotion information reflected by the target input information is more highly correlated with the actual emotion of the target user, the weight of the target input information may be set higher accordingly, thereby improving the accuracy of determining the emotion of the target user when inputting the target input information.

[0116] After determining the emotion of the target user when inputting the target input information, the emotion determination results can be saved in chronological order, and the emotion trend within the predetermined period can be displayed to the target user at some opportunities (such as active request by the user or meeting the push conditions). That is, the emotion trend of the target user within the predetermined period can be determined by the emotion recognition model based on the input information of the target user within the predetermined period.

[0117] The sentiment trend can be displayed in various forms, such as tables, curves, images, text, etc.

[0118] Through the emotional trend, the target user can intuitively view his or her emotions and emotional changes during the predetermined period. The target user can reasonably adjust his or her emotions based on this, and the input method can also reasonably provide corresponding services to the target user based on the emotional trend.

[0119] Figure 2 A device structure diagram of a model determination device for user emotion recognition provided in an embodiment of the present application, the device includes a first determination unit 201, a second determination unit 202, and a training unit 203:

[0120] The first determining unit 201 is used to determine keywords according to input information;

[0121] The second determining unit 202 is used to determine a first type of training samples according to the search results corresponding to the keyword, wherein the first type of training samples includes search content in the search results whose relevance to the keyword meets a condition, and an emotion identifier determined by the search content;

[0122] The training unit 203 is used to train an emotion recognition model according to the first type of training samples, and the emotion recognition model is used to determine the emotion of the target user when the target input information is input according to the target input information of the target user.

[0123] Optionally, the device further includes a third determining unit:

[0124] The third determining unit is used to determine a second type of training samples according to the input behavior data, wherein the second type of training samples includes an input behavior and an emotion identifier corresponding to the input behavior;

[0125] The training unit is further used to train an emotion recognition model according to the first type of training samples and the second type of training samples.

[0126] Optionally, the device further includes a fourth determining unit:

[0127] The fourth determining unit is used to determine a third type of training samples according to the input information, wherein the third type of training samples includes the input information and an emotion identifier corresponding to the input information;

[0128] The training unit is further used to train an emotion recognition model according to the first category of training samples and the third category of training samples.

[0129] Optionally, the device further includes an acquisition unit, a recognition unit and an emotion determination unit:

[0130] The acquisition unit is used to acquire target input information of the target user;

[0131] The recognition unit is used to recognize the emotion identifier corresponding to the target input information according to the emotion recognition model;

[0132] The emotion determination unit is used to determine the emotion of the target user when inputting the target input information according to the target input information and the emotion identifier corresponding to the target input information.

[0133] Optionally, the device further comprises a trend determination unit:

[0134] The trend determination unit is used to determine the emotional trend of the target user within the predetermined time period through the emotion recognition model according to the input information of the target user within the predetermined time period.

[0135] The configuration of each unit or module of the device in the embodiment of the present application can refer to Figure 1 The method shown in the figure is used for implementation and will not be described in detail here.

[0136] It can be seen from the above technical solution that keywords are determined based on the input information generated by the user using the input method, and the determined keywords can obtain corresponding search results through the search engine. According to the search content in the search results, the emotions corresponding to the search content can be analyzed, so the first type of training samples including search content that meets the conditions of keyword relevance and corresponding emotion identifiers can be determined, and the emotion recognition model is trained through the first type of training samples. The emotion recognition model is used to determine the emotion of the target user when the target input information is input according to the target input information of the target user. Since the number of search contents obtained through keyword search is large and carries rich information related to emotions, the emotion identifiers analyzed through the search content not only guarantee accuracy but also have high efficiency. The emotion recognition model trained by a large number of first type training samples with relatively accurate emotion identifiers can accurately identify the user's emotions according to the user's input information.

[0137] See also Figure 3 , is a block diagram of a model determination device for user emotion recognition according to an exemplary embodiment. For example, the device 300 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0138] Reference Figure 3 , the device 300 may include one or more of the following components: a processing component 302 , a memory 304 , a power component 306 , a multimedia component 308 , an audio component 310 , an input / output (I / O) interface 312 , a sensor component 314 , and a communication component 316 .

[0139] The processing component 302 generally controls the overall operation of the device 300, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 302 may include one or more processors 320 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 302 may include one or more modules to facilitate the interaction between the processing component 302 and other components. For example, the processing component 302 may include a multimedia module to facilitate the interaction between the multimedia component 308 and the processing component 302.

[0140] The memory 304 is configured to store various types of data to support operations on the device 300. Examples of such data include instructions for any application or method operating on the device 300, contact data, phone book data, messages, pictures, videos, etc. The memory 304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0141] The power supply component 306 provides power to the various components of the device 300. The power supply component 306 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 300.

[0142] The multimedia component 308 includes a screen that provides an output interface between the device 300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 308 includes a front camera and / or a rear camera. When the device 300 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0143] The audio component 310 is configured to output and / or input audio signals. For example, the audio component 310 includes a microphone (MIC), and when the device 300 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 304 or sent via the communication component 316. In some embodiments, the audio component 310 also includes a speaker for outputting audio signals.

[0144] I / O interface 312 provides an interface between processing component 302 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0145] The sensor assembly 314 includes one or more sensors for providing various aspects of the status assessment of the device 300. For example, the sensor assembly 314 can detect the open / closed state of the device 300, the relative positioning of components, such as the display and keypad of the device 300, the sensor assembly 314 can also detect the position change of the device 300 or a component of the device 300, the presence or absence of user contact with the device 300, the orientation or acceleration / deceleration of the device 300, and the temperature change of the device 300. The sensor assembly 314 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 314 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 314 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0146] The communication component 316 is configured to facilitate wired or wireless communication between the device 300 and other devices. The device 300 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 316 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 316 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0147] In an exemplary embodiment, the apparatus 300 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0148] Specifically, the embodiment of the present invention provides a model determination device 300 for user emotion recognition, including a memory 304 and one or more programs, wherein the one or more programs are stored in the memory 304 and are configured to be executed by one or more processors 320. The one or more programs include instructions for performing the following operations:

[0149] Determine keywords based on input information;

[0150] Determine a first type of training samples according to the search results corresponding to the keyword, wherein the first type of training samples includes search content in the search results whose relevance to the keyword meets a condition, and an emotion identifier determined by the search content;

[0151] An emotion recognition model is trained according to the first type of training samples, and the emotion recognition model is used to determine the emotion of the target user when the target input information is input according to the target input information of the target user.

[0152] Optionally, the processor is further configured to execute the one or more programs including instructions for performing the following operations:

[0153] Determining a second type of training samples according to the input behavior data, wherein the second type of training samples includes the input behavior and the emotion identifier corresponding to the input behavior;

[0154] The step of training the emotion recognition model according to the first type of training samples comprises:

[0155] An emotion recognition model is trained according to the first category of training samples and the second category of training samples.

[0156] Optionally, the processor is further configured to execute the one or more programs including instructions for performing the following operations:

[0157] Determine a third type of training sample according to the input information, wherein the third type of training sample includes the input information and an emotion identifier corresponding to the input information;

[0158] The step of training the emotion recognition model according to the first type of training samples comprises:

[0159] The emotion recognition model is trained according to the first category of training samples and the third category of training samples.

[0160] Optionally, the processor is further configured to execute the one or more programs including instructions for performing the following operations:

[0161] Acquire target input information of the target user;

[0162] Identify the emotion identifier corresponding to the target input information according to the emotion recognition model;

[0163] The emotion of the target user when inputting the target input information is determined according to the target input information and the emotion identifier corresponding to the target input information.

[0164] Optionally, the processor is further configured to execute the one or more programs including instructions for performing the following operations:

[0165] According to the input information of the target user within a predetermined period of time, the emotion trend of the target user within the predetermined period of time is determined by the emotion recognition model.

[0166] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 303 including instructions, and the instructions can be executed by a processor 320 of the apparatus 300 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0167] A machine-readable medium, for example, a non-transitory computer-readable storage medium, when instructions in the medium are executed by a processor of a device (terminal or server), enables the device to execute a model determination method for user emotion recognition, the method comprising:

[0168] Determine keywords based on input information;

[0169] Determine a first type of training samples according to the search results corresponding to the keyword, wherein the first type of training samples includes search content in the search results whose relevance to the keyword meets a condition, and an emotion identifier determined by the search content;

[0170] An emotion recognition model is trained according to the first type of training samples, and the emotion recognition model is used to determine the emotion of the target user when the target input information is input according to the target input information of the target user.

[0171] Figure 4 4 is a schematic diagram of the structure of the server in the embodiment of the present invention. The server 400 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 422 (for example, one or more processors) and memory 432, and one or more storage media 430 (for example, one or more mass storage devices) storing application programs 442 or data 444. Among them, the memory 432 and the storage medium 430 can be short-term storage or permanent storage. The program stored in the storage medium 430 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server. Furthermore, the central processing unit 422 can be configured to communicate with the storage medium 430 to execute a series of instruction operations in the storage medium 430 on the server 400.

[0172] The server 400 may also include one or more power supplies 426, one or more wired or wireless network interfaces 440, one or more input and output interfaces 448, one or more keyboards 446, and / or, one or more operating systems 441, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0173] Those skilled in the art will readily appreciate other embodiments of the invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0174] It should be understood that the present invention is not limited to the precise structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

[0175] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

[0176] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the statement "including one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. The present invention can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0177] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor. The above is only a specific implementation method of the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A model determination method for user emotion recognition, characterized in that: The method is applied to an input method program, and the method comprises: Determine a keyword according to input information, wherein the input information includes content sent by the peer end; According to the keyword, a corresponding search result is obtained by searching in a search engine, and a first type of training samples is determined, wherein the first type of training samples includes search content in the search result that satisfies a condition of relevance to the keyword, and an emotion identifier determined by semantic analysis of the search content; Determining a second type of training sample based on the input behavior data, the second type of training sample including input behaviors and emotion identifiers corresponding to the input behaviors, the input behaviors including withdrawal of input strings, deletion of input content, sentence length in the input content, and input scenarios when the input behaviors are performed; Determine a third type of training sample according to the input information, wherein the third type of training sample includes the input information and an emotion identifier corresponding to the input information; An emotion recognition model is trained according to the first type of training samples, the second type of training samples, and the third type of training samples, wherein the emotion recognition model is used to determine the emotion of the target user when inputting the target input information according to the target input information, the target input behavior, the emotion identifier corresponding to the target input information, and the emotion identifier corresponding to the target input behavior of the target user, wherein the emotion is presented in the form of probability, and the target input information is randomly obtained from the input information of the target user; When the target user inputs input information into the input method program multiple times within a predetermined period of time, the emotion recognition model is used to determine the emotion trend of the target user within the predetermined period of time.

2. A model determination device for user emotion recognition, characterized in that: The device is suitable for use in an input method program, and comprises a first determination unit, a second determination unit, a third determination unit, a fourth determination unit, a training unit, and a trend determination unit: The first determining unit is used to determine a keyword according to input information, where the input information includes content sent by the peer end; The second determination unit is used to determine a first type of training samples by searching the search engine for the keyword to obtain corresponding search results, wherein the first type of training samples includes search content in the search results that satisfies a condition of relevance to the keyword, and an emotion identifier determined by semantic analysis of the search content; The third determination unit is used to determine a second type of training sample according to the input behavior data, wherein the second type of training sample includes an input behavior and an emotion identifier corresponding to the input behavior, wherein the input behavior includes a withdrawal of an input string, a deletion of input content, a sentence length in the input content, and an input scenario when the input behavior is implemented; The fourth determining unit is used to determine a third type of training samples according to the input information, wherein the third type of training samples includes the input information and an emotion identifier corresponding to the input information; The training unit is used to train an emotion recognition model based on the first type of training samples, the second type of training samples and the third type of training samples, wherein the emotion recognition model is used to determine the emotion of the target user when inputting the target input information based on the target input information, the target input behavior, the emotion identifier corresponding to the target input information and the emotion identifier corresponding to the target input behavior of the target user, wherein the emotion is presented in the form of probability, and the target input information is randomly obtained from the input information of the target user; The trend determination unit is used to determine the emotion trend of the target user within the predetermined time period when the target user inputs input information into the input method program multiple times within the predetermined time period, and the emotion recognition model is used to determine the emotion trend of the target user within the predetermined time period.

3. A model determination device for user emotion recognition, the model determination device is suitable for use in an input method program, the model determination device comprises a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the following operations: Determine a keyword according to input information, wherein the input information includes content sent by the peer end; According to the keyword, a corresponding search result is obtained by searching in a search engine, and a first type of training samples is determined, wherein the first type of training samples includes search content in the search result that satisfies a condition of relevance to the keyword, and an emotion identifier determined by semantic analysis of the search content; Determining a second type of training sample based on the input behavior data, the second type of training sample including input behaviors and emotion identifiers corresponding to the input behaviors, the input behaviors including withdrawal of input strings, deletion of input content, sentence length in the input content, and input scenarios when the input behaviors are performed; Determine a third type of training sample according to the input information, wherein the third type of training sample includes the input information and an emotion identifier corresponding to the input information; An emotion recognition model is trained according to the first type of training samples, the second type of training samples, and the third type of training samples, wherein the emotion recognition model is used to determine the emotion of the target user when inputting the target input information according to the target input information, the target input behavior, the emotion identifier corresponding to the target input information, and the emotion identifier corresponding to the target input behavior of the target user, wherein the emotion is presented in the form of probability, and the target input information is randomly obtained from the input information of the target user; When the target user inputs input information into the input method program multiple times within a predetermined period of time, the emotion recognition model is used to determine the emotion trend of the target user within the predetermined period of time.

4. A machine-readable medium having instructions stored thereon, which, when executed by one or more processors, cause an apparatus to perform the method of claim 1.