Emotion recognition method and device in banking business scene

Abstract

The invention provides an emotion recognition method and device in a banking business scene, relates to the technical field of artificial intelligence, and can be used in the fields of meta universe and financial science and technology. The method comprises the following steps: if it is judged that a banking business handled by a client belongs to a specific business, obtaining various biological characteristic data, business channels, business types and historical complaint information of the client; obtaining each biological feature of the customer based on the multiple biological feature data; obtaining weight prediction features; based on each biological feature of the customer and the corresponding emotion recognition model, obtaining an emotion recognition result corresponding to each biological feature of the customer; based on various biological characteristics, weight prediction characteristics and a weight prediction model, obtaining a weight corresponding to each biological characteristic; and obtaining a customer emotion recognition result according to the emotion recognition results corresponding to the various biological characteristics and the respective corresponding weights. According to the emotion recognition method and device in the banking business scene provided by the embodiment of the invention, the accuracy of emotion recognition is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology and can be used in the field of metaverse and financial technology, and specifically to an emotion recognition method and device in a banking business scenario. Background Art

[0002] Emotion recognition can automatically identify an individual's emotional state by acquiring the individual's physiological or non-physiological signals, including facial expressions, voice, heart rate, behavior, text and physiological signal recognition, and judge the user's emotional state through the above data.

[0003] Existing emotion recognition solutions usually rely on a certain biometric feature, such as analyzing facial information, voice information and other biometric features to obtain the user's current emotional state. However, in banking business scenarios, there are situations where the user's current emotional state cannot be accurately reflected. Therefore, how to propose an emotion recognition method that can more accurately identify the customer's emotional state when the customer handles banking business has become an important issue that needs to be solved in this field. Summary of the invention

[0004] In view of the problems in the prior art, an embodiment of the present invention provides an emotion recognition method and device in a banking business scenario, which can at least partially solve the problems in the prior art.

[0005] In a first aspect, the present invention proposes an emotion recognition method in a banking business scenario, comprising:

[0006] If it is determined that the banking business handled by the customer is a specific business, then the multiple biometric data, business channels, business types and customer historical complaint information required for the customer to handle the banking business are obtained;

[0007] Based on the multiple biometric data required by customers for banking services, each biometric feature of the customer is obtained; and the business channel, business type and customer historical complaint information are pre-processed to obtain weighted prediction features;

[0008] Based on each biometric feature of the customer and the corresponding emotion recognition model, obtaining an emotion recognition result corresponding to each biometric feature of the customer; wherein the emotion recognition model corresponding to each biometric feature is obtained by pre-training;

[0009] Based on various biometric features of the customer, the weight prediction features and the weight prediction model, the weight corresponding to each biometric feature is obtained; wherein the weight prediction model is obtained by training based on weight sample data;

[0010] The customer emotion recognition result is obtained based on the emotion recognition results corresponding to the customer's various biometric features and their respective corresponding weights.

[0011] In a second aspect, the present invention provides an emotion recognition device in a banking business scenario, comprising:

[0012] An acquisition module is used to obtain various biometric data, business channels, business types and historical customer complaint information required by the customer to handle banking services after determining that the banking services handled by the customer belong to specific services;

[0013] The acquisition module is used to obtain each biometric feature of the customer based on the multiple biometric data required by the customer to handle banking business; and pre-process the business channel, business type and customer historical complaint information to obtain weight prediction features;

[0014] A first recognition module is used to obtain an emotion recognition result corresponding to each biological feature of the customer based on each biological feature of the customer and a corresponding emotion recognition model; wherein the emotion recognition model corresponding to each biological feature is obtained by pre-training;

[0015] A weight prediction module, used to obtain the weight corresponding to each biometric feature based on various biometric features of the customer, the weight prediction feature and a weight prediction model; wherein the weight prediction model is obtained by training based on weight sample data;

[0016] The second recognition module is used to obtain the customer emotion recognition result according to the emotion recognition results corresponding to various biological characteristics of the customer and their corresponding weights.

[0017] In a third aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the emotion recognition method in the banking business scenario described in any of the above embodiments is implemented.

[0018] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the emotion recognition method in a banking business scenario described in any of the above embodiments.

[0019] In a fifth aspect, the present invention provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the emotion recognition method in the banking business scenario described in any of the above embodiments.

[0020] The emotion recognition method and device in the banking business scenario provided by the embodiment of the present invention, after determining that the banking business handled by the customer belongs to a specific business, obtains multiple biometric data, business channels, business types and customer historical complaint information required for the customer to handle the banking business; based on the multiple biometric data required for the customer to handle the banking business, obtains each biometric feature of the customer; and pre-processes the business channel, business type and customer historical complaint information to obtain weight prediction features; based on each biometric feature of the customer and the corresponding emotion recognition model, obtains the emotion recognition result corresponding to each biometric feature of the customer; based on the various biometric features of the customer, the weight prediction features and the weight prediction model, obtains the weight corresponding to each biometric feature; obtains the customer emotion recognition result based on the emotion recognition results corresponding to the various biometric features of the customer and their respective corresponding weights, and since emotion recognition is performed by integrating multiple biometric features, business information and historical complaint information, the accuracy of emotion recognition is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0022] Figure 1 It is a flowchart of the emotion recognition method in the banking business scenario provided by the first embodiment of the present invention.

[0023] Figure 2 It is a flowchart of the emotion recognition method in the banking business scenario provided by the second embodiment of the present invention.

[0024] Figure 3 It is a flowchart of an emotion recognition method in a banking business scenario provided by the third embodiment of the present invention.

[0025] Figure 4 It is a flowchart of an emotion recognition method in a banking business scenario provided by the fourth embodiment of the present invention.

[0026] Figure 5 It is a flowchart of an emotion recognition method in a banking business scenario provided by the fifth embodiment of the present invention.

[0027] Figure 6 It is a flowchart of an emotion recognition method in a banking business scenario provided by the sixth embodiment of the present invention.

[0028] Figure 7It is a schematic diagram of the structure of an emotion recognition device in a banking business scenario provided by the seventh embodiment of the present invention.

[0029] Figure 8 It is a schematic diagram of the structure of an emotion recognition device in a banking business scenario provided by the eighth embodiment of the present invention.

[0030] Fig. 9 It is a schematic diagram of the structure of an emotion recognition device in a banking business scenario provided by the ninth embodiment of the present invention.

[0031] Fig.10 It is a schematic diagram of the structure of an emotion recognition device in a banking business scenario provided by the tenth embodiment of the present invention.

[0032] Fig.11 It is a structural diagram of an emotion recognition device in a banking business scenario provided by the eleventh embodiment of the present invention.

[0033] Fig.12 It is a schematic diagram of the structure of an emotion recognition device in a banking business scenario provided by the twelfth embodiment of the present invention.

[0034] Fig.13 It is a schematic diagram of the physical structure of an electronic device provided by the thirteenth embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the embodiments of the present invention are further described in detail with reference to the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention. It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other arbitrarily. The acquisition, storage, use, and processing of data in the technical scheme in this application comply with the relevant provisions of laws and regulations. The user information in the embodiments of the present application is obtained through legal and compliant channels, and the acquisition, storage, use, and processing of user information are authorized and agreed by the customer.

[0036] In order to facilitate understanding of the technical solution provided by this application, the relevant contents of the technical solution of this application are first explained below.

[0037] Traditional emotion recognition methods often rely on a certain biometric feature, ignoring the correlation between different biometric features, and do not consider the actual business scenario when performing emotion recognition. Therefore, the embodiment of the present invention proposes an emotion recognition method in a banking business scenario, which collects multiple biometric data such as facial expressions, voice tone and volume, heart rate, blood pressure, and pupil conditions required by customers to handle banking business, and combines multi-dimensional information such as the type of business currently performed by the customer, and comprehensively analyzes the various biometric features of the customer in the banking business scenario to obtain the customer's emotion recognition results, so as to improve the accuracy of customer emotion recognition.

[0038] The following takes the server as an example of the execution subject to illustrate the specific implementation process of the emotion recognition method in the banking business scenario provided by the embodiment of the present invention.

[0039] Figure 1 is a flow chart of an emotion recognition method in a banking business scenario provided by the first embodiment of the present invention. Figure 1 As shown, the emotion recognition method in the banking business scenario provided by the embodiment of the present invention includes:

[0040] S101. If it is determined that the banking business handled by the customer is a specific business, then obtaining various biometric data, business channels, business types and customer historical complaint information required for the customer to handle the banking business;

[0041] Specifically, customers can handle banking services through counters, online banking, mobile banking, self-service terminals, etc. The server will determine whether the banking services handled by the customer belong to specific services. If they belong to specific services, it is necessary to identify the customer's current emotions. The server will obtain various biometric data, business channels and business types required for customers to handle banking services. The server will also obtain customer historical complaint information. Biometric data includes but is not limited to fingerprints, faces, voices, etc., and is collected according to actual business needs. Business channels include but are not limited to counters, self-service terminals, online banking, mobile banking, telephone banking, etc. Customer historical complaint information refers to customer complaints against the bank within a set time period.

[0042] For example, you can specify that certain business types of banking services belong to specific businesses. If the banking services handled by a customer belong to the specified business types, then the banking services handled by the customer belong to the specific businesses.

[0043] S102, based on the multiple biometric data required by the customer for handling banking services, obtaining each biometric feature of the customer; and pre-processing the business channel, business type and customer historical complaint information to obtain weight prediction features;

[0044] Specifically, for each biometric data required by the customer to handle banking services, the server will perform feature processing on each biometric data to obtain each biometric data of the customer. The server will pre-process the business channel, business type and customer historical complaint information to obtain weight prediction features. Pre-processing includes but is not limited to one-hot encoding and feature extraction.

[0045] For example, for a customer's fingerprint data, fingerprint features can be obtained; for a customer's face picture, facial features can be obtained; and for a customer's voice data, voiceprint features can be obtained.

[0046] For example, for business channels and business types, a unique hot coding method can be used. For example, business channels: counter, self-service terminal, online banking, mobile banking and telephone banking are coded as 00001, 00010, 00100, 01000 and 10000 respectively.

[0047] For example, historical customer complaints are usually a text description, which can be converted into numerical information with the help of natural language processing technology, such as keyword extraction, or more complex sentiment analysis. For example, the frequency of complaints can be recorded, or the main issues of complaints can be analyzed as service attitude, business errors, etc., and this information can be encoded.

[0048] S103, based on each biological feature of the customer and the corresponding emotion recognition model, obtaining an emotion recognition result corresponding to each biological feature of the customer; wherein the emotion recognition model corresponding to each biological feature is obtained by pre-training;

[0049] Specifically, the server inputs each biometric feature of the customer into the emotion recognition model corresponding to each biometric feature, and can obtain the emotion recognition result corresponding to each biometric feature of the customer. The emotion recognition result corresponding to each biometric feature can include the probability of the customer being in each emotional state. The emotion recognition model corresponding to each biometric feature is obtained by pre-training.

[0050] For example, based on a customer's face picture, facial expression features can be obtained, and an emotion recognition model corresponding to the facial expression features can be trained through the facial expression features.

[0051] Based on the customer's voice data, voiceprint features can be obtained, and the emotion recognition model corresponding to the voiceprint features can be trained through the voiceprint features.

[0052] You can also train the corresponding emotion recognition models separately through physiological indicators such as heart rate, breathing rate, blood pressure, skin conductance, muscle tension, body temperature, etc.

[0053] S104, based on various biometric features of the customer, the weight prediction features and the weight prediction model, obtaining a weight corresponding to each biometric feature; wherein the weight prediction model is obtained by training based on weight sample data;

[0054] Specifically, the server inputs various biometric features of the client and the weight prediction features into a weight prediction model, and can output the weight corresponding to each biometric feature of the client. The weight prediction model is obtained by training based on weight sample data.

[0055] S105 , obtaining a customer emotion recognition result according to the emotion recognition results corresponding to various biological characteristics of the customer and their corresponding weights.

[0056] Specifically, the server can obtain the customer emotion recognition result according to the emotion recognition results corresponding to various biological characteristics of the customer and their corresponding weights.

[0057] For example, the emotion recognition results A, B and C corresponding to the three biometric features of the customer are obtained, and the corresponding weights are a, b and c respectively. The emotion recognition results corresponding to each biometric feature include the probability that the customer is in five different emotional states. For an emotional state, the probability in the emotion recognition result A is x, the probability in the emotion recognition result B is y, and the probability in the emotion recognition result C is z, then the total probability of the emotional state P1 = ax + by + cz. Similarly, the total probabilities of the other four emotional states can be obtained as P2, P3, P4 and P5 respectively. Compare P1, P2, P3, P4 and P5, and take the emotional state corresponding to the maximum value as the customer's emotion recognition result.

[0058] The emotion recognition method in the banking business scenario provided by the embodiment of the present invention obtains multiple biometric data, business channels, business types and customer historical complaint information required for the customer to handle banking business after determining that the banking business handled by the customer belongs to a specific business; obtains each biometric feature of the customer based on the multiple biometric data required for the customer to handle banking business; and pre-processes the business channel, business type and customer historical complaint information to obtain weight prediction features; obtains emotion recognition results corresponding to each biometric feature of the customer based on each biometric feature of the customer and the corresponding emotion recognition model; obtains the weight corresponding to each biometric feature based on the various biometric features of the customer, the weight prediction features and the weight prediction model; obtains the customer emotion recognition results based on the emotion recognition results corresponding to the various biometric features of the customer and their respective corresponding weights. Since emotion recognition is performed by integrating multiple biometric features, business information and historical complaint information, the accuracy of emotion recognition is improved.

[0059] Figure 2is a flow chart of an emotion recognition method in a banking business scenario provided by a second embodiment of the present invention. Figure 2 As shown, on the basis of the above embodiments, further, obtaining a weight prediction model based on weight sample data training includes:

[0060] S201, obtaining weighted sample data, wherein the weighted sample data includes a plurality of data samples, each of which includes various biometric training data and weighted feature training data of a customer handling banking services;

[0061] Specifically, historical data corresponding to banking services can be collected, and weighted sample data can be obtained from the historical data corresponding to banking services. The weighted sample data includes multiple data samples. Each data sample includes various biometric training data and weighted feature training data of customers handling banking services. Emotion labels can include satisfied, normal, and dissatisfied, which are set according to actual needs and are not limited in the embodiments of the present invention.

[0062] S202: According to each data sample and the original model, a weight prediction model is obtained through training.

[0063] Specifically, the server trains the original model based on the weight sample data to obtain the weight prediction model. The original model can adopt a neural network model, which is selected according to actual needs and is not limited in the embodiment of the present invention. The specific training process of the model is prior art and will not be described here.

[0064] Figure 3 is a flow chart of an emotion recognition method in a banking business scenario provided by a third embodiment of the present invention. Figure 3 As shown, based on the above embodiments, further, obtaining weighted sample data includes:

[0065] S301. Based on the historical data corresponding to the banking services, obtain various biometric data, business channels and business types of the customer's banking services, and obtain historical complaint information corresponding to the banking services;

[0066] Specifically, historical data corresponding to banking services can be collected, that is, historical data of various banking services handled by various customers, including various biometric data, business channels, and business types provided by customers when handling banking services. From the historical data corresponding to banking services, various biometric data, business channels, and business types of banking services handled by customers can be obtained. Complaint information of various banking services by various customers in historical time periods can be collected as historical complaint information corresponding to banking services. Among them, the historical time period is set according to actual needs, and the embodiment of the present invention does not limit it.

[0067] S302, based on each biometric data of the customer handling banking services, obtaining each biometric training data of each customer; pre-processing the business channel, business type and customer historical complaint information of each customer handling banking services, and obtaining weighted feature training data of each customer;

[0068] Specifically, the server performs feature processing on each biometric feature data of the customer handling banking services, and can obtain each biometric feature training data of each customer. The server performs preprocessing on the business channel, business type and customer historical complaint information of each customer handling banking services, and can obtain weight feature training data of each customer.

[0069] S303: Use various biometric feature training data and weight feature training data of each customer as a data sample of the weight sample data.

[0070] Specifically, the server regards various biometric training data and weighted feature training data of each client as a data sample, and each data sample constitutes weighted sample data.

[0071] Figure 4 is a flow chart of an emotion recognition method in a banking business scenario provided by a fourth embodiment of the present invention. Figure 4 As shown, based on the above embodiments, further, if it is determined that the banking business handled by the customer belongs to a specific business, it includes:

[0072] S401, obtaining the biometric type and banking business data corresponding to the banking business handled by the customer;

[0073] Specifically, when a customer handles banking services, the server can obtain the type of biometric features corresponding to the banking services handled by the customer, and can also obtain banking service data corresponding to the banking services handled by the customer. Banking service data includes but is not limited to service type, customer service completion status, service amount, transaction frequency, etc.

[0074] For example, for banking services, the business codes of each category of banking services can be traversed to find the code blocks related to biometric identification, such as biometric data processing and calling. In the code blocks, the code blocks related to biometric identification are usually marked with specific functions or keywords. By counting the specific functions or keywords related to biometric identification, the types of biometrics corresponding to banking services can be counted, that is, how many types of biometrics are required to handle the banking services, and the counted types of biometrics corresponding to the banking services are stored in correspondence with the banking services.

[0075] S402, if it is determined that the number of biometric characteristics corresponding to the banking business handled by the customer is greater than 1, then obtaining a risk prediction result of the banking business handled by the customer based on the banking business data and a risk prediction model; wherein the risk prediction model is obtained by training based on the bank's historical business data and corresponding risk labels;

[0076] Specifically, the server determines whether the type of biometric features corresponding to the banking business handled by the customer is greater than 1. If the type of biometric features corresponding to the banking business handled by the customer is greater than 1, the risk of the banking business is predicted based on the banking business data and the risk prediction model to obtain the risk prediction result of the banking business handled by the customer. The risk prediction result is a risky business or a non-risky business. The risk prediction model is obtained by training based on the bank's historical business data and the corresponding risk labels.

[0077] S403. If the risk prediction result of the banking business handled by the customer is a risky business, it is determined that the banking business handled by the customer belongs to a specific business.

[0078] Specifically, if the risk prediction result of the banking business handled by the customer is a risky business, then the banking business handled by the customer belongs to a specific business and the customer's emotion recognition needs to be performed. For non-risk businesses, emotion recognition of the customer may not be performed to speed up the processing efficiency of banking business.

[0079] Through the risk prediction model, it is possible to determine which businesses have certain risks and require the bank to pay attention to the customer situation without manually formulating rules. As new businesses emerge as the bank system is updated, it is also possible to identify whether the new businesses are risky businesses.

[0080] Figure 5 is a flow chart of an emotion recognition method in a banking business scenario provided by a fifth embodiment of the present invention. Figure 5 As shown, on the basis of the above embodiments, further, the risk prediction model is obtained based on the bank's historical business data and the corresponding risk label training, including:

[0081] S501, obtaining the bank's historical business data and corresponding risk labels;

[0082] Specifically, the bank's historical business data may be collected, including business type, customer business completion status, business amount, transaction frequency and other data, which may be selected according to actual needs and are not limited in the embodiments of the present invention. The bank's historical business data may include multiple business sample data and risk labels corresponding to each business sample data, and the risk labels may be obtained by manual annotation.

[0083] For example, if the amount of a banking transaction is greater than a set amount and is a payment transaction, it can be marked as a risky transaction.

[0084] S502: Based on the bank's historical business data and the initial model, a risk prediction model is trained to obtain the risk prediction model.

[0085] Specifically, the server performs model training based on the bank's historical business data and the initial model, and can train a risk prediction model. The initial model can adopt a deep neural network model. The specific training process of the model is prior art and will not be described here.

[0086] Among them, the key features of the bank's historical business data can be extracted by using an autoencoder or a clustering algorithm, etc. For example, similar risk features may exist for all payment transactions with an amount exceeding a certain amount.

[0087] Figure 6 is a flow chart of an emotion recognition method in a banking business scenario provided by a sixth embodiment of the present invention. Figure 6 As shown, based on the above embodiments, before obtaining the various biometric data required by the customer for handling banking services, it also includes:

[0088] S601, receiving a hardware list reported by a client;

[0089] Specifically, before collecting multiple biometric data of the customer, the client can report a hardware list to the server, and the server can receive the hardware list. The hardware list includes but is not limited to cameras, microphones, fingerprint sensors, etc. The client includes but is not limited to mobile terminals, desktop computers, self-service terminals, etc.

[0090] S602: If the hardware devices in the hardware list do not match the multiple biometric features required by the customer to handle banking services, a prompt message indicating that biometric feature collection hardware is missing is sent to the client.

[0091] Specifically, the server determines whether the hardware devices in the hardware list match the multiple biometric features required by the customer to handle banking services, that is, whether the multiple biometric data required by the customer to handle banking services can be collected through the hardware devices in the hardware list. If the hardware devices in the hardware list do not match the multiple biometric features required by the customer to handle banking services, it means that all the biometric data required by the customer to handle banking services cannot be obtained, and the server will send a prompt message to the client that the biometric collection hardware is missing, so as to prompt the customer that the client currently used cannot handle the business.

[0092] By determining whether the client's hardware matches the biometric features required for banking services, timely reminders will be given for customers who are unable to conduct banking services due to hardware reasons, thereby improving the customer's business experience.

[0093] On the basis of the above embodiments, further, the emotion recognition method in the banking business scenario provided by the embodiment of the present invention also includes:

[0094] The customer emotion recognition result is synchronized to the backend system.

[0095] Specifically, after obtaining the customer emotion recognition result, the server may synchronize the customer emotion recognition result to a background system, so that the background system can use the customer emotion recognition result to provide better services to customers.

[0096] For example, the banking industry has launched various functions with Metaverse tags on mobile clients, creating exclusive roles for customers to conduct business in virtual space. However, since customers and customer service in the Metaverse are virtual roles, and virtual roles are not associated with customer emotions, and since people cannot communicate face to face, it is more difficult for bank customer service staff to identify negative emotions such as anxiety and dissatisfaction in customers, which leads to customer complaints due to the inability to provide timely assistance. Synchronizing customer emotion recognition results to the backend system of the Metaverse service and writing customer emotion recognition results into the customer Metaverse role can help customer service staff in the Metaverse identify customer emotions through the Metaverse role and improve services.

[0097] Figure 7 is a schematic diagram of the structure of an emotion recognition device in a banking business scenario provided by the seventh embodiment of the present invention. Figure 7 As shown, the emotion recognition device in the banking business scenario provided by the embodiment of the present invention includes an acquisition module 701, an acquisition module 702, a first recognition module 703, a weight prediction module 704 and a second recognition module 705, wherein:

[0098] The acquisition module 701 is used to obtain various biometric data, business channels, business types and customer historical complaint information required by the customer to handle banking business after determining that the banking business handled by the customer belongs to a specific business; the acquisition module 702 is used to obtain each biometric feature of the customer based on the various biometric data required by the customer to handle banking business; and pre-process the business channels, business types and customer historical complaint information to obtain weight prediction features; the first recognition module 703 is used to obtain the emotion recognition results corresponding to each biometric feature of the customer based on each biometric feature of the customer and the corresponding emotion recognition model; wherein the emotion recognition model corresponding to each biometric feature is pre-trained; the weight prediction module 704 is used to obtain the weight corresponding to each biometric feature based on the various biometric features of the customer, the weight prediction features and the weight prediction model; wherein the weight prediction model is obtained through training based on weight sample data; the second recognition module 705 is used to obtain the customer's emotion recognition results based on the emotion recognition results corresponding to the various biometric features of the customer and their respective corresponding weights.

[0099] Specifically, customers can handle banking services through counters, online banking, mobile banking, self-service terminals, etc. The acquisition module 701 will determine whether the banking services handled by the customer belong to specific services. If they belong to specific services, the customer's current emotions need to be identified. The acquisition module 701 will obtain various biometric data, business channels, and business types required for customers to handle banking services. The acquisition module 701 will also obtain customer historical complaint information. Biometric data includes but is not limited to fingerprints, faces, voices, etc., and is collected according to actual business needs. Business channels include but are not limited to counters, self-service terminals, online banking, mobile banking, telephone banking, etc. Customer historical complaint information refers to customer complaints against the bank within a set time period.

[0100] For each biometric data required by the customer to handle banking business, the acquisition module 702 will perform feature processing on each biometric data to obtain each biometric data of the customer. The acquisition module 702 will pre-process the business channel, business type and customer historical complaint information to obtain weight prediction features. The pre-processing includes but is not limited to unique hot encoding and feature extraction.

[0101] The first recognition module 703 inputs each biometric feature of the customer into the emotion recognition model corresponding to each biometric feature, and can obtain the emotion recognition result corresponding to each biometric feature of the customer. The emotion recognition result corresponding to each biometric feature may include the probability of the customer being in each emotional state. The emotion recognition model corresponding to each biometric feature is obtained by pre-training.

[0102] The weight prediction module 704 inputs various biometric features of the customer and the weight prediction features into a weight prediction model, and can output the weight corresponding to each biometric feature of the customer. The weight prediction model is obtained by training based on weight sample data.

[0103] The second recognition module 705 can obtain the customer emotion recognition result according to the emotion recognition results corresponding to various biological characteristics of the customer and their corresponding weights.

[0104] The emotion recognition method and device in the banking business scenario provided by the embodiment of the present invention, after determining that the banking business handled by the customer belongs to a specific business, obtains multiple biometric data, business channels, business types and customer historical complaint information required for the customer to handle the banking business; based on the multiple biometric data required for the customer to handle the banking business, obtains each biometric feature of the customer; and pre-processes the business channel, business type and customer historical complaint information to obtain weight prediction features; based on each biometric feature of the customer and the corresponding emotion recognition model, obtains the emotion recognition result corresponding to each biometric feature of the customer; based on the various biometric features of the customer, the weight prediction features and the weight prediction model, obtains the weight corresponding to each biometric feature; obtains the customer emotion recognition result based on the emotion recognition results corresponding to the various biometric features of the customer and their respective corresponding weights, and since emotion recognition is performed by integrating multiple biometric features, business information and historical complaint information, the accuracy of emotion recognition is improved.

[0105] Figure 8 is a schematic diagram of the structure of an emotion recognition device in a banking business scenario provided by an eighth embodiment of the present invention. Figure 8 As shown, based on the above embodiments, further, the emotion recognition device in the banking business scenario provided by the embodiment of the present invention further includes a first sample acquisition module 706 and a first training module 707, wherein:

[0106] The first sample acquisition module 706 is used to obtain weight sample data, and the weight sample data includes multiple data samples, each data sample includes various biometric training data and weight feature training data of customers handling banking services; the first training module 707 is used to train and obtain a weight prediction model based on each data sample and the original model.

[0107] Fig. 9 is a schematic diagram of the structure of an emotion recognition device in a banking business scenario provided by a ninth embodiment of the present invention. Fig. 9 As shown, based on the above embodiments, further, the first sample obtaining module 706 includes a first obtaining unit 7061, a second obtaining unit 7062 and an acting unit 7063, wherein:

[0108] The first acquisition unit 7061 is used to obtain various biometric data, business channels and business types of customers handling banking services based on historical data corresponding to banking services, and obtain historical complaint information corresponding to banking services; the second acquisition unit 7062 is used to obtain each biometric training data of each customer based on each biometric data of the customer handling banking services; pre-process the business channel, business type and historical complaint information corresponding to the banking services of each customer to obtain weighted feature training data of each customer; and the unit 7063 is used to use various biometric training data and weighted feature training data of each customer as a data sample of the weighted sample data.

[0109] Fig.10 is a schematic diagram of the structure of an emotion recognition device in a banking business scenario provided by the tenth embodiment of the present invention. Fig.10 As shown, based on the above embodiments, further, the acquisition module 701 includes an acquisition unit 7011, a judgment unit 7012 and a determination unit 7013, wherein:

[0110] The acquisition unit 7011 is used to obtain the biometric type and banking business data corresponding to the banking business handled by the customer; the judgment unit 7012 is used to obtain the risk prediction result of the banking business handled by the customer based on the banking business data and the risk prediction model if it is judged that the biometric type corresponding to the banking business handled by the customer is greater than 1; wherein the risk prediction model is obtained based on the bank's historical business data and corresponding risk label training; the determination unit 7013 is used to determine that the banking business handled by the customer belongs to a specific business if the risk prediction result of the banking business handled by the customer is a risky business.

[0111] Fig.11 is a schematic diagram of the structure of an emotion recognition device in a banking business scenario provided by the eleventh embodiment of the present invention. Fig.11 As shown, based on the above embodiments, further, the emotion recognition device in the banking business scenario provided by the embodiment of the present invention further includes a second sample acquisition module 708 and a second training module 709, wherein:

[0112] The second sample acquisition module 708 is used to obtain the bank's historical business data and corresponding risk labels; the second training module 709 is used to train and obtain a risk prediction model based on the bank's historical business data and the initial model.

[0113] Fig.12 is a schematic diagram of the structure of an emotion recognition device in a banking business scenario provided by the twelfth embodiment of the present invention. Fig.12As shown, based on the above embodiments, further, the emotion recognition device in the banking business scenario provided by the embodiment of the present invention further includes a receiving module 710 and a prompting module 711, wherein:

[0114] The receiving module 710 is used to receive the hardware list reported by the client; the prompt module 711 is used to send a prompt message to the client indicating that the biometric collection hardware is missing if the hardware devices in the hardware list do not match the multiple biometrics required by the customer to handle banking services.

[0115] On the basis of the above embodiments, further, the emotion recognition device in the banking business scenario provided by the embodiment of the present invention further includes a synchronization module, wherein:

[0116] The synchronization module is used to synchronize the customer emotion recognition result to the background system.

[0117] The server embodiment provided in the embodiment of the present invention can be specifically used to execute the processing flow of the above-mentioned method embodiments. Its functions are not described in detail here, and reference can be made to the detailed description of the above-mentioned method embodiments.

[0118] It should be noted that the emotion recognition method and device in the banking business scenario provided by the embodiment of the present invention can be used in the financial field, and can also be used in any technical field other than the financial field. The embodiment of the present invention does not limit the application field of the emotion recognition method and device in the banking business scenario.

[0119] Fig.13 is a schematic diagram of the physical structure of an electronic device provided by the thirteenth embodiment of the present invention, such as Fig.13As shown, the electronic device may include: a processor (processor) 1301, a communication interface (Communications Interface) 1302, a memory (memory) 1303 and a communication bus 1304, wherein the processor 1301, the communication interface 1302, and the memory 1303 communicate with each other through the communication bus 1304. Processor 1301 can call the logic instructions in memory 1303 to execute the following method: if it is determined that the banking business handled by the customer belongs to a specific business, then obtain multiple biometric data, business channels, business types and customer historical complaint information required for the customer to handle banking business; based on the multiple biometric data required for the customer to handle banking business, obtain each biometric of the customer; and pre-process the business channel, business type and customer historical complaint information to obtain weight prediction features; based on each biometric of the customer and the corresponding emotion recognition model, obtain the emotion recognition result corresponding to each biometric of the customer; wherein the emotion recognition model corresponding to each biometric is obtained by pre-training; based on the various biometrics of the customer, the weight prediction features and the weight prediction model, obtain the weight corresponding to each biometric; wherein the weight prediction model is obtained based on weight sample data training; according to the emotion recognition results corresponding to the various biometrics of the customer and their respective corresponding weights, obtain the customer's emotion recognition results.

[0120] In addition, the logic instructions in the above-mentioned memory 1303 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0121] The present embodiment discloses a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the methods provided by the above-mentioned method embodiments, for example, including: if it is determined that the banking business handled by the customer belongs to a specific business, then obtaining multiple biometric data, business channels, business types and customer historical complaint information required for the customer to handle the banking business; based on the multiple biometric data required for the customer to handle the banking business, obtaining each biometric of the customer; and pre-processing the business channels, business types and customer historical complaint information to obtain weight prediction features; based on each biometric of the customer and the corresponding emotion recognition model, obtaining an emotion recognition result corresponding to each biometric of the customer; wherein the emotion recognition model corresponding to each biometric is obtained by pre-training; based on the various biometrics of the customer, the weight prediction features and the weight prediction model, obtaining the weight corresponding to each biometric; wherein the weight prediction model is obtained by training based on weight sample data; according to the emotion recognition results corresponding to the various biometrics of the customer and their respective corresponding weights, obtaining the customer emotion recognition result.

[0122] The present embodiment provides a computer-readable storage medium, which stores a computer program, and the computer program enables the computer to execute the methods provided by the above-mentioned method embodiments, for example, including: if it is determined that the banking business handled by the customer belongs to a specific business, then obtaining multiple biometric data, business channels, business types and customer historical complaint information required for the customer to handle the banking business; based on the multiple biometric data required for the customer to handle the banking business, obtaining each biometric of the customer; and pre-processing the business channels, business types and customer historical complaint information to obtain weight prediction features; based on each biometric of the customer and the corresponding emotion recognition model, obtaining an emotion recognition result corresponding to each biometric of the customer; wherein the emotion recognition model corresponding to each biometric is obtained by pre-training; based on the various biometrics of the customer, the weight prediction features and the weight prediction model, obtaining the weight corresponding to each biometric; wherein the weight prediction model is obtained based on weight sample data training; according to the emotion recognition results corresponding to the various biometrics of the customer and their respective corresponding weights, obtaining the customer emotion recognition results.

[0123] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0124] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0125] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0126] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0127] In the description of this specification, the description with reference to the terms "one embodiment", "a specific embodiment", "some embodiments", "for example", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0128] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for emotion recognition in a banking business scenario, characterized in that: include: If it is determined that the banking business handled by the customer is a specific business, then the multiple biometric data, business channels, business types and customer historical complaint information required for the customer to handle the banking business are obtained; Based on the multiple biometric data required by customers for banking services, each biometric feature of the customer is obtained; and the business channel, business type and customer historical complaint information are pre-processed to obtain weighted prediction features; Based on each biometric feature of the customer and the corresponding emotion recognition model, obtaining an emotion recognition result corresponding to each biometric feature of the customer; wherein the emotion recognition model corresponding to each biometric feature is obtained by pre-training; Based on various biometric features of the customer, the weight prediction features and the weight prediction model, the weight corresponding to each biometric feature is obtained; wherein the weight prediction model is obtained by training based on weight sample data; The customer emotion recognition result is obtained based on the emotion recognition results corresponding to the customer's various biometric features and their respective corresponding weights.

2. The method according to claim 1, characterized in that The weight prediction model obtained by training based on weight sample data includes: Obtaining weighted sample data, wherein the weighted sample data includes a plurality of data samples, each of which includes various biometric training data and weighted feature training data of a customer handling banking services; According to each data sample and the original model, a weight prediction model is trained.

3. The method according to claim 2, characterized in that The obtaining of weighted sample data comprises: Based on the historical data corresponding to banking services, obtain various biometric data, business channels and business types of customers handling banking services, and obtain historical complaint information corresponding to banking services; Based on each biometric data of the customer handling banking business, obtain each biometric training data of each customer; pre-process the business channel, business type and historical complaint information corresponding to the banking business of each customer to obtain the weighted feature training data of each customer; The various biometric training data and weighted feature training data of each customer are used as a data sample of the weighted sample data.

4. The method according to claim 1, characterized in that: If it is determined that the banking business handled by the customer is a specific business, it includes: Obtain the biometric type and banking business data corresponding to the banking business handled by the customer; If it is determined that the number of biometric characteristics corresponding to the banking business handled by the customer is greater than 1, then obtaining the risk prediction result of the banking business handled by the customer based on the banking business data and the risk prediction model; wherein the risk prediction model is obtained based on the bank's historical business data and the corresponding risk label training; If the risk prediction result of the banking business conducted by the customer is a risky business, it is determined that the banking business conducted by the customer belongs to a specific business.

5. The method according to claim 4, characterized in that The risk prediction models obtained by training based on the bank's historical business data and corresponding risk labels include: Obtain the bank's historical business data and corresponding risk labels; Based on the bank's historical business data and the initial model, a risk prediction model is trained.

6. The method according to claim 1, characterized in that Before obtaining the various biometric data required by customers for banking services, it also includes: Receive the hardware list reported by the client; If the hardware devices in the hardware list do not match the multiple biometric features required by the customer to handle banking services, a prompt message indicating that the biometric feature collection hardware is missing is sent to the client.

7. The method according to any one of claims 1 to 6, characterized in that: Also includes: The customer emotion recognition result is synchronized to the backend system.

8. An emotion recognition device in a banking business scenario, characterized in that: include: An acquisition module is used to obtain various biometric data, business channels, business types and historical customer complaint information required by the customer to handle banking services after determining that the banking services handled by the customer belong to specific services; The acquisition module is used to obtain each biometric feature of the customer based on the multiple biometric data required by the customer to handle banking business; and pre-process the business channel, business type and customer historical complaint information to obtain weight prediction features; A first recognition module is used to obtain an emotion recognition result corresponding to each biological feature of the customer based on each biological feature of the customer and a corresponding emotion recognition model; wherein the emotion recognition model corresponding to each biological feature is obtained by pre-training; A weight prediction module, used to obtain the weight corresponding to each biometric feature based on various biometric features of the customer, the weight prediction feature and a weight prediction model; wherein the weight prediction model is obtained by training based on weight sample data; The second recognition module is used to obtain the customer emotion recognition result according to the emotion recognition results corresponding to various biological characteristics of the customer and their corresponding weights.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images