User emotion recognition method, electronic equipment, medium and product

Abstract

The invention discloses a user emotion recognition method, electronic equipment, a medium and a product, and relates to the technical field of data processing. According to the method, the behavior emotion recognition result is generated based on the user behavior data matched with the actual business scene, and compared with a scheme of recognizing emotion according to images in a traditional scheme, the behavior data for emotion recognition can be more easily obtained, the behavior data can directly reflect the emotion condition of the user, and the user experience is improved. A large number of training samples and computing power do not need to be adopted for training, so that the implementation cost is lower. And on the basis of the behavior recognition result, a target emotion recognition result is obtained by fusing a physiological emotion recognition result generated through the user voice, so that the accuracy of the target emotion recognition result is ensured. Therefore, according to the embodiment of the invention, the emotion recognition scheme can be more conveniently applied to the actual communication service scene under the condition of ensuring the accuracy of the recognition result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a user emotion recognition method, an electronic device, a medium and a product. BACKGROUND

[0002] In the production and operation process, the transaction payment behavior of the user often has a certain correlation with the user's emotion. When the user is accompanied by positive emotion during transaction payment, it often means that the user has a good sensory experience of the business transaction, and the probability of continuous business transaction is also larger. Conversely, it usually means that the user will no longer continue to carry out the corresponding business. At present, the technical solutions for identifying emotions are mostly based on user facial images to identify expressions and then obtain the user's emotion. However, in some scenarios in actual application, it is difficult to obtain the user's facial image, such as a scenario of communicating only with the user's voice, which causes that the current emotion recognition scheme may be difficult to apply to actual business scenarios.

[0003] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0004] The main purpose of the present application is to provide a user emotion recognition method, an electronic device, a medium and a product, which aims to solve the technical problem that the current emotion recognition scheme is difficult to apply to actual business scenarios.

[0005] To achieve the above purpose, the present application provides a user emotion recognition method, which comprises the following steps:

[0006] determining a basic emotion sequence and a basic emotion change frequency sequence of a user corresponding emotion observation interval based on user behavior data;

[0007] generating a behavior emotion recognition result of the user based on the basic emotion sequence and the basic emotion change frequency sequence;

[0008] generating a physiological emotion recognition result of the user based on business voice interaction data corresponding to the emotion observation interval;

[0009] fusing the behavior emotion recognition result and the physiological emotion recognition result to obtain a target emotion recognition result of the user.

[0010] Optionally, the step of determining a basic emotion sequence and a basic emotion change frequency sequence of a user corresponding emotion observation interval based on user behavior data comprises:

[0011] extending a preset time period in a historical time direction based on a current time to obtain the emotion observation interval;

[0012] Determine each observation node within the stated emotion observation interval;

[0013] Based on user behavior data, the basic sentiment values ​​of each observation node are extracted to form a basic sentiment sequence, and the number of sentiment changes in each sub-interval obtained by segmenting each observation node is extracted to form a basic sentiment change frequency sequence.

[0014] Optionally, the steps of extracting basic sentiment values ​​from each observation node based on user behavior data to form a basic sentiment sequence, and extracting the number of sentiment changes in each sub-interval obtained from each observation node to form a basic sentiment change frequency sequence include:

[0015] For any one of the observation nodes, obtain the user behavior data of the target sub-interval corresponding to the observation node;

[0016] The user's multi-indicator multi-dimensional features are extracted from the user behavior data to obtain the basic sentiment value of the observation node, wherein the multi-dimensional features include at least one of proximity features, frequency features, and value features;

[0017] After traversing each observation node, the basic sentiment values ​​of each observation node are combined into the basic sentiment sequence in chronological order.

[0018] For any one of the sub-intervals, the difference between the basic sentiment values ​​of the corresponding two endpoints of the sub-interval is taken as the number of basic sentiment changes in the sub-interval.

[0019] After traversing each sub-interval, the number of basic emotional changes in each sub-interval is combined into a sequence of basic emotional change counts in chronological order.

[0020] Optionally, the step of generating the user's behavioral emotion recognition result based on the basic emotion sequence and the basic emotion change frequency sequence includes:

[0021] The first emotion fluctuation value is generated by accumulating the fluctuations of each basic emotion value in the basic emotion sequence relative to adjacent basic emotion values.

[0022] A second emotion fluctuation value is generated by accumulating the fluctuation of each emotion change frequency in the basic emotion change frequency sequence relative to a preset change frequency threshold.

[0023] The behavioral emotion recognition result is generated based on the proportional relationship between the first emotion fluctuation value and the second emotion fluctuation value.

[0024] Optionally, the step of generating the user's physiological emotion recognition result based on the business voice interaction data corresponding to the emotion observation interval includes:

[0025] User features are extracted based on the business voice interaction data, wherein the user features include at least one of the following: user speaking volume, user speaking frequency, keyword repetition, and call response speed.

[0026] The user characteristics are input into a preset recognition model to generate the user's physiological emotion recognition result.

[0027] Optionally, the step of fusing the behavioral emotion recognition result and the physiological emotion recognition result to obtain the user's target emotion recognition result includes:

[0028] The target emotion recognition result is obtained by weighted summation of the behavioral emotion recognition result and the physiological emotion recognition result based on preset weights. The preset weights are obtained by fitting the emotion calculation results of each labeled sample. The emotion calculation results include the behavioral emotion results and physiological emotion results of each labeled sample.

[0029] Optionally, after the step of fusing the behavioral emotion recognition result and the physiological emotion recognition result to obtain the user's target emotion recognition result, the method includes:

[0030] The target emotion recognition result is used as a prompt word, and the prompt word and the user's business profile are input into a preset large language model to generate business text for communicating with the user.

[0031] In addition, to achieve the above objectives, this application also proposes an electronic device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the user emotion recognition method as described above.

[0032] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the user emotion recognition method described above.

[0033] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the user emotion recognition method described above.

[0034] One or more technical solutions proposed in this application have at least the following technical effects:

[0035] In this embodiment, the basic emotion sequence and the sequence of basic emotion changes within the corresponding emotion observation interval are determined based on the user's behavioral data. Based on the basic emotion sequence and the sequence of basic emotion changes, a behavioral emotion recognition result is generated for the user. Based on the business voice interaction data corresponding to the emotion observation interval, a physiological emotion recognition result is generated for the user. The behavioral emotion recognition result and the physiological emotion recognition result are then fused to obtain the user's target emotion recognition result. That is, in this embodiment, the user's basic emotion and the sequence of basic emotion changes are determined based on the user's behavioral data, and a behavioral emotion recognition result is generated based on these two emotion sequences. Then, a physiological emotion recognition result is generated based on the voice data. Finally, the behavioral emotion recognition result and the physiological emotion recognition result are fused to obtain the target emotion recognition result.

[0036] It is understood that the embodiments of this application will generate behavioral emotion recognition results based on user behavior data that aligns with actual business scenarios. Compared to traditional solutions that identify emotions based on images, this application can more easily obtain behavioral data for emotion recognition, and the behavioral data can directly reflect the user's emotional state, eliminating the need for a large number of training samples and computational power, thus reducing implementation costs. Furthermore, this application will integrate physiological emotion recognition results generated from user voice to obtain the target emotion recognition result, ensuring the accuracy of the target emotion recognition result. Therefore, the embodiments of this application can more conveniently apply the emotion recognition solution to actual communication business scenarios while ensuring the accuracy of the recognition results. Attached Figure Description

[0037] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0038] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart illustrating the first embodiment of the user emotion recognition method in this application;

[0040] Figure 2 This is a schematic diagram of the overall framework of the user emotion recognition method in this application;

[0041] Figure 3 This is a schematic diagram of the full-scale fusion model in the user emotion recognition method of this application;

[0042] Figure 4 This is a schematic diagram illustrating the application framework of the user emotion recognition method in this application;

[0043] Figure 5 This is a clustering diagram in the user emotion recognition method of this application;

[0044] Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the user emotion recognition method in this application embodiment.

[0045] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0046] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0047] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0048] In the course of production and operation, users' transaction and payment behaviors are often inevitably linked to their emotions. When users experience positive emotions during transactions, it usually means that they have a good impression of the transaction, and the probability of the transaction continuing is higher. Conversely, it usually means that users will not continue to engage in the corresponding business. However, most current emotion recognition technologies rely on facial images to identify facial expressions and thus determine the user's emotions. This makes it difficult to apply current emotion recognition solutions to real-world business scenarios. On the one hand, in actual communication business scenarios, user facial images are not inherently compatible with communication services, making it difficult to obtain user image data. On the other hand, recognizing user emotions through facial images is costly, requiring manual annotation of millions of facial micro-expression images and extensive computing resources for data training. Data acquisition is difficult, annotation costs are very high (requiring a large number of highly specialized psychology and micro-expression recognition experts for data annotation), and the time required is very long (requiring significant computing power and training time for millions of images). The algorithm's accuracy is also limited.

[0049] The main solution of this application embodiment is: determining the basic emotion sequence and the basic emotion change frequency sequence of the user's corresponding emotion observation interval based on the user's behavioral data; generating the user's behavioral emotion recognition result based on the basic emotion sequence and the basic emotion change frequency sequence; generating the user's physiological emotion recognition result based on the business voice interaction data corresponding to the emotion observation interval; and fusing the behavioral emotion recognition result and the physiological emotion recognition result to obtain the user's target emotion recognition result.

[0050] In other words, this application's embodiments generate behavioral emotion recognition results based on user behavior data that aligns with actual business scenarios. Compared to traditional solutions that identify emotions from images, this application can more easily obtain behavioral data for emotion recognition, and this behavioral data can directly reflect the user's emotional state, eliminating the need for extensive training samples and computational power, thus reducing implementation costs. Furthermore, this application integrates physiological emotion recognition results generated from user voice into the behavioral recognition results to obtain the target emotion recognition result, ensuring the accuracy of the target emotion recognition result. Therefore, this application's embodiments can more conveniently apply the emotion recognition solution to actual communication business scenarios while ensuring the accuracy of the recognition results.

[0051] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a cloud platform, computer, mobile phone, etc., or an electronic device capable of realizing the above functions.

[0052] Based on the above description, this application provides a user emotion recognition method, referring to... Figure 1 This is a flowchart illustrating the first embodiment of the user emotion recognition method of this application.

[0053] In this embodiment, the user emotion recognition method includes steps S10 to S40:

[0054] Step S10: Determine the basic emotion sequence and the sequence of basic emotion changes in the corresponding emotion observation interval based on the user's behavioral data;

[0055] It should be noted that, in this embodiment, the execution entity of the aforementioned user emotion recognition method can be a server deployed in the cloud. Besides recognizing user emotions, this server can also provide users with corresponding communication services and record user consumption and payment data related to these services, thereby providing basic data support for emotion recognition. Furthermore, since the emotion recognition process is essentially the same for each user, this embodiment will use one user as an example for explanation.

[0056] For example, the aforementioned user behavior data may include user consumption behavior data and payment behavior data. It is understood that user consumption behavior data and payment behavior data can, to some extent, reflect the positive or negative nature of a user's emotions. For instance, the more positive a user's emotions, the more actively they participate in consumption and payment. The aforementioned user emotion observation interval typically refers to the time period for observing user emotions. The specific length of the emotion observation interval can be set by technical personnel according to actual needs, such as several months (e.g., one month), fifteen days, or one week, etc. Based on the user's consumption and payment data generated within the emotion observation interval, a basic emotion sequence and a basic emotion change frequency sequence are extracted. The basic emotion sequence consists of the basic emotion values ​​corresponding to each observation node within the emotion observation interval, while the basic emotion change frequency sequence consists of the number of emotion changes corresponding to each sub-interval formed by dividing the observation nodes. It should be noted that each of the aforementioned observation nodes can evenly divide the emotion observation interval into sub-intervals of equal length. Each sub-interval contains at least one observation node at each of its two endpoints, and each observation node corresponds to one sub-interval. Whether an observation node corresponds to its left or right sub-interval can be set by technical personnel; no specific restrictions are imposed here. Furthermore, the base emotion value of each observation node is extracted from the user's consumption and payment data within the corresponding sub-interval. The number of emotion changes within a sub-interval is obtained from the difference between the base emotion values ​​at the two endpoints of that sub-interval.

[0057] Step S20: Based on the basic emotion sequence and the basic emotion change frequency sequence, generate the user's behavioral emotion recognition result;

[0058] It should be noted that the basic emotional sequence and the basic emotional change frequency sequence obtained above can both reflect the user's emotional fluctuations. Therefore, the user's emotion recognition results can be generated through the basic emotional sequence and the basic emotional change frequency sequence.

[0059] For example, a user's behavioral emotion recognition result can be calculated using a basic emotion sequence and a basic emotion change frequency sequence. The greater the fluctuation reflected by the basic emotion sequence and the basic emotion change frequency sequence, the higher the value of the behavioral emotion recognition result, indicating that the user's basic emotion tends towards the negative side; conversely, the smaller the fluctuation reflected by the basic emotion sequence and the basic emotion change frequency sequence, the lower the value of the behavioral emotion recognition result, indicating that the user's basic emotion tends towards the positive side.

[0060] Step S30: Generate the user's physiological emotion recognition result based on the business voice interaction data corresponding to the emotion observation interval;

[0061] For example, the aforementioned emotion observation interval may contain not only user consumption or payment data, but also user business voice interaction data, such as when a user calls customer service. Correspondingly, user voice data more directly reflects user emotions to a certain extent. Therefore, user physiological emotion recognition results can be generated using user business voice interaction data. For instance, in application, voice features can be extracted from the voice interaction data, and then input into a pre-trained emotion recognition model to generate physiological emotion recognition results.

[0062] Step S40: Fuse the behavioral emotion recognition result and the physiological emotion recognition result to obtain the user's target emotion recognition result.

[0063] For example, the behavioral emotion recognition results and physiological emotion recognition results obtained in the above steps are fused. The fusion method can be a weighted sum of the behavioral and physiological emotion recognition results, or it can be a direct average of the two. After fusing the emotion recognition results obtained from both methods, the user's target emotion recognition result is obtained. Similarly, the larger the value of the target emotion recognition result, the more negative the user's basic emotion tends to be; the smaller the value of the target emotion recognition result, the more positive the user's basic emotion tends to be. It is understandable that the final target emotion recognition result can be used to guide customer service. For example, when a customer service representative is preparing to recommend a product to a user, they can determine the user's emotional tendency based on the target emotion recognition result, and then decide whether to recommend the product or how to recommend it. It is worth noting that the specific application of the target emotion recognition result in telecommunications industry service scenarios can be set by the user according to actual needs, so no restrictions are imposed here.

[0064] In this embodiment, the basic emotion sequence and the sequence of basic emotion changes within the corresponding emotion observation interval for the user will be determined based on the user's behavioral data. Based on the basic emotion sequence and the sequence of basic emotion changes, a behavioral emotion recognition result for the user will be generated. Based on the business voice interaction data corresponding to the emotion observation interval, a physiological emotion recognition result for the user will be generated. Finally, the behavioral emotion recognition result and the physiological emotion recognition result will be fused to obtain the user's target emotion recognition result. That is, in this embodiment, the user's sequences in terms of basic emotion and the number of basic emotion changes will be determined based on the user's behavioral data, and a behavioral emotion recognition result will be generated based on these two emotion sequences. Then, a physiological emotion recognition result will be generated based on the voice data. Finally, the behavioral emotion recognition result and the physiological emotion recognition result will be fused to obtain the target emotion recognition result.

[0065] It is understood that the embodiments of this application will generate behavioral emotion recognition results based on user behavior data that aligns with actual business scenarios. Compared to traditional solutions that identify emotions based on images, this application can more easily obtain behavioral data for emotion recognition, and the behavioral data can directly reflect the user's emotional state, eliminating the need for a large number of training samples and computational power, thus reducing implementation costs. Furthermore, this application will integrate physiological emotion recognition results generated from user voice to obtain the target emotion recognition result, ensuring the accuracy of the target emotion recognition result. Therefore, the embodiments of this application can more conveniently apply the emotion recognition solution to actual communication business scenarios while ensuring the accuracy of the recognition results.

[0066] In one feasible implementation, the step of determining the basic emotion sequence and the basic emotion change frequency sequence for the user's corresponding emotion observation interval based on the user's behavioral data includes steps S11 to S13:

[0067] Step S11: Based on the current moment, extend a preset time period in the direction of historical time to obtain the emotion observation interval;

[0068] Step S12: Determine each observation node in the emotion observation interval;

[0069] Step S13: Extract the basic emotion values ​​of each observation node based on the user's behavior data to form a basic emotion sequence, and extract the number of emotion changes in each sub-interval obtained by segmenting each observation node to form a basic emotion change frequency sequence.

[0070] For example, when determining the emotion observation interval, a preset time period can be extended from the current moment towards historical time or the past to obtain the emotion observation interval. The preset time period can be set by technicians according to actual needs, such as one month, half a month, or one week. Next, each observation node within the emotion interval is determined, and the number or interval of each observation node can be determined based on a preset number of observation nodes. Each observation node can divide the emotion observation area into uniform sub-intervals. After determining each observation node, a basic emotion value can be extracted from the user's behavioral data. This basic emotion value can be determined through behavioral data. For example, behavioral data can include user consumption or payment behavior data, including the amount consumed, the frequency of consumption, and the date of the most recent consumption. These data are then statistically integrated to form a basic emotion value, and correspondingly, each basic emotion value can be combined to form a basic emotion sequence. Similarly, the number of emotion changes is extracted from each sub-interval. The extracted number of emotion changes forms a basic emotion change frequency sequence, and the number of emotion changes in a sub-interval can be determined based on the difference in the basic emotion values ​​at the two endpoints of that sub-interval.

[0071] In a feasible implementation, the steps of extracting basic emotion values ​​from each observation node based on user behavior data to form a basic emotion sequence, and extracting the number of emotion changes in each sub-interval obtained by segmenting each observation node to form a basic emotion change frequency sequence, include steps S131 to S135:

[0072] Step S131: For any one of the observation nodes, obtain the user behavior data of the target sub-interval corresponding to the observation node;

[0073] Step S132: Extract multi-dimensional features of the user's multi-indicators from the user behavior data to obtain the basic sentiment value of the observation node, wherein the multi-dimensional features include at least one of proximity features, frequency features, and value features;

[0074] Step S133: After traversing each observation node, the basic sentiment values ​​of each observation node are combined into the basic sentiment sequence in chronological order.

[0075] Step S134: For any one of the sub-intervals, the difference between the basic emotion values ​​of the corresponding two endpoints of the sub-interval is taken as the number of basic emotion changes in the sub-interval.

[0076] Step S135: After traversing each sub-interval, the basic emotional change frequency of each sub-interval is combined into a basic emotional change frequency sequence according to the time order.

[0077] For example, since the processing for determining each observation node is basically similar, this embodiment will use one observation node as an example for explanation. For any one of the observation nodes, user behavior data for the target sub-interval corresponding to that observation node is obtained. It can be understood that the sub-interval corresponding to an observation node is the target sub-interval, and the user behavior data for the target sub-interval refers to the data generated by the user's consumption or payment behavior within that target sub-interval. Then, multi-dimensional features of the user's multiple indicators are extracted from the user behavior data, and these extracted multi-dimensional features can be used as the basic sentiment value of the observation node. Here, multiple indicators can refer to consumption and payment, or to different pre-set product types, such as C-end product indicators, H-end product indicators, or N-end product indicators. The multi-dimensional features can include R-dimensional, F-dimensional, and M-dimensional features, which respectively refer to user proximity data (the distance between the payment event and the current time when the consumption event occurs, i.e., proximity feature), user frequency data (i.e., frequency feature), and user value data (the magnitude of the value, i.e., value feature). For example:

[0078] User recent data refers to data on users' most recent payments and service usage. Examples of recently paid data include the number of months since the last subscription in the last three months, the number of months since the last phone purchase in the last three years, and the number of months since the last package subscription in the last three months. Examples of recently used service data include the number of days since the last data usage in the current month, the number of days since the last outgoing call in the current month, and the number of months since the last phone change in the last three years.

[0079] User frequency data includes data on the frequency of user payments and the frequency of user service usage. Specifically, user payment frequency data includes, for example, the number of times tariffs have changed in the past three months, the number of times handsets have been purchased in the past three years, and the number of times bundled services have been purchased in the past three months. User service usage frequency data includes, for example, the number of days of data usage in the current month, the number of days of outgoing calls in the current month, and the number of times handsets have been changed in the past three years.

[0080] User value metrics data includes user-paid value metrics data and service usage value metrics data. Among them, user-paid value metrics data includes, for example, the cost of the package in the past three months, the amount of credit-based mobile phone purchases in the past three years, and the cost of the bundled service in the past three months; user service usage value metrics data includes, for example, the current month's data usage saturation, the current month's outgoing call saturation, and the current month's bundled data usage saturation.

[0081] Furthermore, considering both consumption and payment, the specific metrics for C-end products, H-end products, and N-end products can be summarized as follows:

[0082] C-end product metrics explanation:

[0083] Consumption type: Number of days since last data usage in the current month, Number of days since last outgoing call in the current month, Number of times the most recent phone change in the past three years to this month, Number of days of data usage in the current month, Number of days of outgoing calls in the current month, Number of phone changes in the past three years, Data usage saturation in the current month, Outgoing call saturation in the current month, Data usage saturation of the current month's data plan, Total price of credit-based phone purchases in the past three years.

[0084] Payment type: Number of months since the most recent package application in the past three months, Number of months since the last purchase date in the past three years, Number of days since the most recent package application date in the past three months, Number of tariff changes in the past three months, Number of purchases in the past three years, Number of package applications in the past three months, Package fees in the past three months, Package fees in the past three months, Package fees in the past three months, Amount of credit purchases in the past three years.

[0085] H-end product indicator explanation:

[0086] Consumption type: Number of days since last online activity in the past three months, number of days since last online security activity in the past three months, number of days since last broadband usage in the past three months, number of days since last TV viewing in the past three months, number of days since last VIP program viewing in the past three months, number of days of broadband usage in the past three months, number of TV on-demand sessions in the past three months, number of days of VIP program viewing in the past three months, network online time in the past three months, security online time in the past three months, user bandwidth in the past three months, viewing time in the past three months, number of VIP program episodes viewed in the past three months.

[0087] Payment Types: Number of times network setup was accepted within the last three months, number of times security services were accepted within the last three months, number of times broadband services were accepted within the last three months, number of times TV services were accepted within the last three months, number of times large-screen memberships were accepted within the last three months, number of times network setup was processed within the last three months, number of times security services were processed within the last three months, number of times broadband services were processed within the last three months, number of times TV services were processed within the last three months, number of times large-screen memberships were processed within the last three months, network setup activation fees within the last three months, security activation fees within the last three months, broadband activation fees within the last three months, TV activation fees within the last three months, and large-screen membership activation fees within the last three months.

[0088] N-end product metrics description:

[0089] Consumption type: Number of days since the last redemption of membership benefits in the past three months, Number of days since the most recent visit to the portal app (Application) in the past three months, Number of days since the most recent visit to the Caiyun app in the past three months, Number of times membership benefits were redeemed in the past three months, Number of days of visits to the portal app in the past three months, Number of days of visits to the Caiyun app in the past three months, Value of membership benefits redeemed in the past three months, Traffic to the portal app in the past three months, Traffic to the Caiyun app in the past three months.

[0090] Payment Types: Number of times the most recently accepted integration package application was processed in the past three months; Number of times the most recently accepted benefits package application was processed in the past three months; Number of times the most recently accepted portal app application was processed in the past three months; Number of times the most recently accepted HeCaiYun application was processed in the past three months; Number of times the most recently accepted integration package application was processed in the past three months; Number of times the most recently accepted benefits package application was processed in the past three months; Number of times the most recently accepted HeCaiYun application was processed in the past three months; Fees for integration packages activated in the past three months; Fees for benefits packages in the past three months; Fees for portal apps in the past three months; Fees for HeCaiYun in the past three months.

[0091] It should be noted that the multi-dimensional features of users mentioned above, based on the examples above, can be set by the users themselves and will not be elaborated further here. Furthermore, to facilitate data processing, the data extracted into the RFM category can be normalized. For example, the IQR (Interquartile Range) method can be used to determine extreme values ​​for the consumption and payment data in the RFM category. The upper limit is calculated as upper = Q3 + 3 * IQR. If a data point is greater than upper, it is considered an extreme value, and the extreme value is replaced by the upper value (where IQR = Q3 - Q1). After performing a log(x+1) transformation on the data, normalization is performed according to the extreme values. The transformation formula is as follows:

[0092] For example, the normalization formula for proximity index can be: 1 - (x - min(x)) / (max(x) - min(x)); the normalization formula for frequency and value index is: (x - min(x)) / (max(x) - min(x)). In the formula, max(x) and min(x) refer to the maximum and minimum values ​​determined by the IQR method, respectively, and x represents the data point.

[0093] After traversing each observation node through the above process, the basic sentiment value corresponding to each observation node can be obtained. Then, the basic sentiment values ​​are combined in chronological order to obtain the basic sentiment sequence.

[0094] Similarly, since the processing for each sub-region is basically the same, this embodiment will take one sub-region as an example. That is, for any sub-region, the difference between the basic sentiment values ​​at its two endpoints is taken as the number of basic sentiment changes in that sub-region. For example, the difference can be obtained by subtracting the basic sentiment values ​​at the two endpoints. The endpoints of the sub-region are actually the observation nodes from which the sub-region is segmented. Then, by combining the number of basic sentiment changes in each sub-region in chronological order, the sequence of the number of basic sentiment changes can be obtained.

[0095] For reference Figure 2 This is a schematic diagram of the overall framework in this application embodiment. R-index (dimension) data, F-index (dimension) data, and M-index (dimension) data will be input into the data collection and integration module for normalization and generation of two sequences, etc. The basic emotion model training module can be used to train the user's voice emotion recognition model. The emotion-corresponding data input to this module can refer to the emotions labeled in the original data and can be used for training. The model generation module is used to generate the user's emotion recognition results through the model. The model usage module is used to apply the emotion recognition results generated by the model to customer service.

[0096] Furthermore, in this embodiment, a full-scale fusion module may also be provided, as shown in reference...Figure 3 This is a schematic diagram of the full-scale fusion model. The full-scale fusion module mainly merges the output results (behavioral emotion recognition results) of the fusion model (mainly fusion of paid RFM model and consumer RFM model) with the output results (physiological emotion recognition results) of the NLP model (used to recognize user voice data) to obtain the final target emotion recognition result, namely Score_silent basic emotion in the figure.

[0097] In one feasible implementation, the step of generating the user's behavioral emotion recognition result based on the basic emotion sequence and the basic emotion change frequency sequence includes steps S21 to S23:

[0098] Step S21: Accumulate the fluctuation of each basic emotion value in the basic emotion sequence relative to adjacent basic emotion values ​​to generate a first emotion fluctuation value;

[0099] Step S22: Accumulate the fluctuation of each emotion change frequency in the basic emotion change frequency sequence relative to a preset change frequency threshold to generate a second emotion fluctuation value;

[0100] Step S23: Generate the behavioral emotion recognition result based on the proportional relationship between the first emotion fluctuation value and the second emotion fluctuation value.

[0101] For example, a first emotion fluctuation value is generated by accumulating the fluctuations of each basic emotion value in the basic emotion sequence relative to adjacent basic emotion values. For instance, the first emotion fluctuation value is obtained by accumulating the differences of each basic emotion value relative to the previous basic emotion value (i.e., the fluctuations of each basic emotion value relative to adjacent basic emotion values). Similarly, a second emotion fluctuation value is obtained by accumulating the fluctuations of each emotion change frequency in the basic emotion change frequency sequence relative to a preset change frequency threshold. The preset change frequency threshold can have a maximum change threshold and a minimum change threshold. For any given emotion change frequency, the magnitude of that frequency relative to the maximum and minimum change thresholds can be used as the fluctuation of that emotion change frequency. After obtaining the first and second emotion fluctuation values, the above-mentioned behavioral emotion recognition result can be calculated based on the proportional relationship between the first and second emotion fluctuation values. Further, in a feasible implementation, the formula for generating the behavioral emotion recognition result can be as follows:

[0102]

[0103] In the formula, M represents the emotion recognition result of the behavior, and D... r This refers to the base sentiment value for the r-th observation node, and correspondingly, D r-1 This refers to the base sentiment value of the (r-1)th observation node (i.e., its neighboring base sentiment values). This is the accumulated first emotional fluctuation value; R j,min F j,min M j,min and R j,max F j,max M j,max These are the minimum and maximum change thresholds for the j-th sub-interval, respectively, which can be set by technical personnel based on experience. j F j M j This represents the number of basic emotional changes in the j-th sub-interval. This is the accumulated second emotional fluctuation value.

[0104] In a feasible implementation, the step of generating the user's physiological emotion recognition result based on the business voice interaction data corresponding to the emotion observation interval includes steps S31 to S32:

[0105] Step S31: Extract user features based on the business voice interaction data, wherein the user features include at least one of user speaking volume, user speaking frequency, keyword repetition, and call response speed;

[0106] Step S32: Input the user features into the preset recognition model to generate the user's physiological emotion recognition result.

[0107] It should be noted that, in this embodiment, in order to ensure the accuracy of the final target emotion recognition result, the user's voice data will also be integrated to identify the user's emotions.

[0108] For example, business voice interaction data generated by users within an emotion observation interval or sub-interval can be obtained. This business voice interaction data typically refers to the voice data generated when users communicate with customer service. User features are then extracted from this business voice data. These features include at least one of the following: user speaking volume, user speaking frequency, keyword repetition, and call response speed. In practical applications, to improve accuracy, user features can also include all of the above features. After extracting the user features, they are input into a pre-trained recognition model, i.e., the aforementioned preset recognition model, to generate the user's physiological emotion recognition result. The preset recognition model can be a RoBERTa model. In practical applications, a large number of labeled voice samples can be used to train the initial model. For example, the user's speaking volume, frequency, repeated keywords, and response speed during the call can be used as key features, and then categorized as negative, neutral, and positive according to {-1, 0, 1}. Negative indicates negative emotions, neutral indicates ambiguous emotions, and positive indicates positive emotions. These voice training samples can cover more news, community Q&A, encyclopedia, and other content, enabling the model to learn richer language representations. In this embodiment, the learning rate of the recognition model can be set to 2e-5, and the optimizer can be trained using the AdamW method. The trained initial model is then used as the preset recognition model. Furthermore, it is worth noting that when training the preset recognition model, the labeling of the training samples can be further refined. That is, in addition to negative and positive emotions, classifications of negative and positive levels can be added to enrich the output of the preset recognition model. Finally, a preset transformation relationship can be used to quantify the output of the preset recognition model to obtain physiological emotion recognition results, which can then be fused with behavioral emotion recognition results to correct the behavioral emotion recognition results.

[0109] In one feasible implementation, the step of fusing the behavioral emotion recognition result and the physiological emotion recognition result to obtain the user's target emotion recognition result includes step S41:

[0110] Step S41: The target emotion recognition result is obtained by weighted summation of the behavioral emotion recognition result and the physiological emotion recognition result based on preset weights. The preset weights are obtained by fitting the emotion calculation results of each labeled sample. The emotion calculation results include the behavioral emotion results and physiological emotion results of each labeled sample.

[0111] For example, after obtaining the behavioral emotion recognition result and the physiological emotion recognition result, the two emotion recognition results can be weighted and summed using preset weights to obtain the target emotion recognition result. It should be noted that the preset weights can be obtained by fitting the emotion calculation results of each labeled sample, and the emotion calculation results include the behavioral emotion results and physiological emotion results of each labeled sample. For example, each labeled sample can be a customer who has conducted business within a certain period and a customer who has not conducted business, and each is assigned a different customer label. Then, the behavioral emotion results and physiological emotion results of the labeled customers (i.e., the labeled samples) are calculated according to the method used in the above embodiment for calculating the emotion recognition results and physiological emotion recognition results. Finally, with the goal of being able to distinguish whether a user tends to conduct business based on the fusion result of the behavioral emotion results and physiological emotions, the emotion calculation results are fitted to obtain the respective weights (i.e., the preset weights) of the target emotion recognition result (the behavioral emotion recognition result and the physiological emotion recognition result).

[0112] For example, in one feasible implementation, a prior hypothesis can be manually designed, assuming that the weights of the two emotion recognition results are [0.2, 0.8]. Then, the actual weights are determined using the target formula. and These are the weights for the action emotion recognition result and the physiological emotion recognition result, respectively, and the target formula is as follows:

[0113]

[0114] In the formula, m on the left side of the equation represents the target emotion recognition result, tanh on the right side is the activation function, and softmax on the right side is used for binary classification regression. In practical applications, the result obtained after training... It can be 0.6, It can be 0.4.

[0115] Additionally, it should be noted that the target emotion recognition result obtained in the above process can be the final determination of the user's positive or negative emotion, such as comparing the weighted sum with a threshold to determine whether the user's emotion is negative or positive, or it can simply be the quantitative result of the weighted sum.

[0116] In one feasible implementation, after the step of fusing the behavioral emotion recognition result and the physiological emotion recognition result to obtain the user's target emotion recognition result, the method includes step S50:

[0117] Step S50: Use the target emotion recognition result as a prompt word, and input the prompt word and the user's business profile into a preset large language model to generate business text for communicating with the user.

[0118] For example, in this embodiment, the target emotion recognition result is also applied to customer service. The target emotion recognition result is used as a prompt word, and the prompt word and the user's business profile are input into a preset large language model. The preset large language model then generates business text for communication with the user based on the prompt word and the user's business profile. Correspondingly, the customer service robot or human customer service representative can communicate with the user by referring to the business text. In practical applications, a task objective can also be input into the preset large language model; for example, the task objective could be business promotion or a follow-up call. (See reference...) Figure 4 This diagram illustrates the application framework used in this application. The model module will be integrated with the business system and the large language model to complement the customer service component within the business system. Compared to purely human customer service, this embodiment can deliver more standardized marketing or communication scripts, provide 24 / 7 customer service, and reduce the cost of understanding customers for human customer service representatives.

[0119] Furthermore, the aforementioned business profiles can be obtained by aggregating a large amount of consumption and payment data from different users. This allows for the identification of each user's consumption and payment type, enabling targeted business recommendations. For example, suppose the normalized dataset in the above context is (X1, X2, ..., X...). n ), where each data point X i It is a d-dimensional vector (d = 68 in this scheme). We need to divide these data points into K (K = 9 in this scheme) clusters {C_1, C_2, ..., C_9}. The optimization objective of K-means can be expressed as:

[0120]

[0121] In the formula, U K It refers to C k Cluster center point. This formula also represents the optimization objective as minimizing the distance between each data point X and the cluster center of its respective cluster.

[0122] Reference Figure 5This diagram illustrates the clustering process in this application embodiment. Users can be categorized into nine consumption and payment types: low-consumption, low-payment; medium-consumption, low-payment; high-consumption, low-payment; low-consumption, medium-payment; medium-consumption, medium-payment; high-consumption, medium-payment; high-consumption, low-payment; high-consumption, medium-payment; and high-payment, high-payment. In the actual clustering results, the payment score center for low-payment users is located around 0.13 in the d-dimensional vector space, for medium-payment users around 0.33, and for high-payment users around 0.62. The consumption type is determined by the payment type; even among high-consumption users, the consumption score center for low-payment, high-consumption users is significantly higher than the payment score center, while the consumption score center for high-payment, high-consumption users is slightly higher than the payment score center. This classification method effectively identifies the payment sentiment of users, providing a useful reference for business marketing and promotion. Different marketing sentiments lead to different product recommendations. For example, "high-end" users are recommended high-consumption, high-quality packages; "value-for-money" users are recommended mid-range packages and combinations of promotional offers.

[0123] The following is for reference. Figure 6 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, mobile terminals such as computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as computers. Figure 6 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0124] like Figure 6As shown, the electronic device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the electronic device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although electronic devices with various systems are shown in the figures, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0125] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0126] The electronic device provided in this application, employing the user emotion recognition method described in the above embodiments, can solve the technical problem that current emotion recognition schemes are difficult to apply to actual business scenarios. Compared with the prior art, the beneficial effects of the electronic device provided in this application are the same as those of the user emotion recognition method provided in the above embodiments, and other technical features of the electronic device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0127] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0128] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0129] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the user emotion recognition method in the above embodiments.

[0130] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0131] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0132] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by an electronic device, cause the electronic device to:

[0133] Based on user behavior data, determine the basic emotion sequence and the basic emotion change frequency sequence for the corresponding emotion observation interval of the user; based on the basic emotion sequence and the basic emotion change frequency sequence, generate the user's behavioral emotion recognition result; based on the business voice interaction data corresponding to the emotion observation interval, generate the user's physiological emotion recognition result; fuse the behavioral emotion recognition result and the physiological emotion recognition result to obtain the user's target emotion recognition result.

[0134] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0135] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0136] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0137] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described user emotion recognition method. This solves the technical problem that current emotion recognition schemes are difficult to apply to real-world business scenarios. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the user emotion recognition method provided in the above embodiments, and will not be repeated here.

[0138] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the user emotion recognition method described above.

[0139] The computer program product provided in this application can solve the technical problem of user emotion recognition. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the user emotion recognition method provided in the above embodiments, and will not be repeated here.

[0140] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A user emotion recognition method, characterized in that, The user emotion recognition method includes the following steps: Based on the user's behavioral data, determine the basic emotion sequence and the basic emotion change frequency sequence for the corresponding emotion observation interval of the user. Based on the basic emotion sequence and the basic emotion change frequency sequence, the user's behavioral emotion recognition result is generated; Based on the business voice interaction data corresponding to the emotion observation interval, the physiological emotion recognition result of the user is generated; By fusing the behavioral emotion recognition results and the physiological emotion recognition results, the target emotion recognition result of the user is obtained.

2. The user emotion recognition method as described in claim 1, characterized in that, The steps of determining the basic emotion sequence and the basic emotion change frequency sequence for the user's corresponding emotion observation interval based on the user's behavioral data include: The emotion observation interval is obtained by extending a preset time period from the current moment towards historical time. Determine each observation node within the stated emotion observation interval; Based on user behavior data, the basic sentiment values ​​of each observation node are extracted to form a basic sentiment sequence, and the number of sentiment changes in each sub-interval obtained by segmenting each observation node is extracted to form a basic sentiment change frequency sequence.

3. The user emotion recognition method as described in claim 2, characterized in that, The steps of extracting basic sentiment values ​​from each observation node based on user behavior data to form a basic sentiment sequence, and extracting the number of sentiment changes in each sub-interval obtained from each observation node to form a basic sentiment change frequency sequence include: For any one of the observation nodes, obtain the user behavior data of the target sub-interval corresponding to the observation node; The user's multi-indicator multi-dimensional features are extracted from the user behavior data to obtain the basic sentiment value of the observation node, wherein the multi-dimensional features include at least one of proximity features, frequency features and value features; After traversing each observation node, the basic sentiment values ​​of each observation node are combined into the basic sentiment sequence in chronological order. For any one of the sub-intervals, the difference between the basic sentiment values ​​of the corresponding two endpoints of the sub-interval is taken as the number of basic sentiment changes in the sub-interval. After traversing each sub-interval, the number of basic emotional changes in each sub-interval is combined into a sequence of basic emotional change counts in chronological order.

4. The user emotion recognition method as described in claim 1, characterized in that, The step of generating the user's behavioral emotion recognition result based on the basic emotion sequence and the basic emotion change frequency sequence includes: The first emotion fluctuation value is generated by accumulating the fluctuations of each basic emotion value in the basic emotion sequence relative to adjacent basic emotion values. A second emotion fluctuation value is generated by accumulating the fluctuation of each emotion change frequency in the basic emotion change frequency sequence relative to a preset change frequency threshold. The behavioral emotion recognition result is generated based on the proportional relationship between the first emotion fluctuation value and the second emotion fluctuation value.

5. The user emotion recognition method as described in claim 1, characterized in that, The step of generating the user's physiological emotion recognition result based on the business voice interaction data corresponding to the emotion observation interval includes: User features are extracted based on the business voice interaction data, wherein the user features include at least one of the following: user speaking volume, user speaking frequency, keyword repetition, and call response speed. The user characteristics are input into a preset recognition model to generate the user's physiological emotion recognition result.

6. The user emotion recognition method as described in claim 1, characterized in that, The step of fusing the behavioral emotion recognition result and the physiological emotion recognition result to obtain the user's target emotion recognition result includes: The target emotion recognition result is obtained by weighted summation of the behavioral emotion recognition result and the physiological emotion recognition result based on preset weights. The preset weights are obtained by fitting the emotion calculation results of each labeled sample. The emotion calculation results include the behavioral emotion results and physiological emotion results of each labeled sample.

7. The user emotion recognition method as described in claim 1, characterized in that, After the step of fusing the behavioral emotion recognition result and the physiological emotion recognition result to obtain the user's target emotion recognition result, the method includes: The target emotion recognition result is used as a prompt word, and the prompt word and the user's business profile are input into a preset large language model to generate business text for communicating with the user.

8. An electronic device, characterized in that, The electronic device includes: a processor, a memory, and a user emotion recognition method program stored in the memory and executable on the processor, wherein the user emotion recognition method program, when executed, implements the steps of the user emotion recognition method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a user emotion recognition method program, which, when executed, implements the steps of the user emotion recognition method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes a user emotion recognition method program, which, when executed by a processor, implements the steps of the user emotion recognition method as described in any one of claims 1 to 7.

Images